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ABSTRACT 


This paper reports the results of a study that compared the effectiveness of the 
traditional school-house and an interacttve multimedia system in training Repair Party 
Leaders. The study was conducted over a period of three months at the Repair Party 
Leader School, Fleet Training Center, San Diego, California. Subjects participating in 
the study ranged from mid-grade enlisted sailors (second class petty officers) to junior 
officers (Ensign through Lieutenant Commander). 

Findings from three separate analyses are presented. The data analysis provides a 
basis for concluding that the interactive system may be an effective way to deliver 
training to members of the shipboard damage control organization in a manner in keeping 
with the “Smart Ship” concept. Exit level of knowledge comparisons between personnel 
that used the IDCTT (RPL) as a supplement to the standard RPL school curriculum and 
those that did not are examined, and final results and recommendations are provided. 
Recommendations for improvement of the system, as well as further research topics are 


also provided. 
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EXECUTIVE SUMMARY 


Over the past several years, computer-based, interactive training has become an 
important part of the military’s education system. This technology, once out of reach due 
to cost, is gaining acceptance within the Department of the Navy as a means of either 
supplementing or replacing school-based training. With the decreasing cost of Windows- 
based personal computers, the Navy is aggressively pursuing development and 
procurement of interactive courseware that will, by providing inoproved, “Just in time” 
training, enhance the overall combat readiness of ships and aircraft within the force. 

The IDCTT (RPL) is a U.S. Navy-developed software program that provides 
realistic integrated shipboard casualty control training. This system utilizes a large 
database and numerous video clips to provide the trainee with a very realistic training 
environment. The intent of IDCTT (RPL) is to provide the student with practice in 
collecting and processing information, coordinating responses, and maintaining the 
overall damage control picture. These skills are difficult to develop outside of actually 
imposing damage control scenarios and then evaluating the Repair Party Leader’s actions 
and responses. IDCTT provides the RPL candidate with unlimited opportunities to “test 
his or her mettle” against the computer, practicing personnel and equipment management, 
prioritization of effort and event sequencing. This software also provides the support 
necessary to conduct simultaneous interactive team training. The ability to modify 
scenarios at the local command level enables the ship’s damage control training team to 
tailor training to specifically target areas of weakness identified during drills or other 
training sessions. IDCTT (RPL) provides immediate student performance evaluation and 
feedback, reinforcing the training received by the user. Finally, cross-platform 
connectivity between Windows-, Unix- and OS2-based operating systems allows for 
system flexibility. 

Repair Party Leaders are trained at Fleet Training Centers in San Diego, 
California, and Norfolk, Virginia. The course of mstruction emphasizes advanced 


damage control theory through classroom lecture, and provides for some practical 


Xi 


application of topics presented in the classroom. However, the limited nature of the 
course, as well as restricted facilities, allow the student little opportunity to actually 
coordinate the activities of a Repair Locker, manage personnel and equipment, or 
exercise decision-making techniques. It is thought that use of IDCTT (RPL) as an 
educational augment at the school will enhance this opportunity, and result in an 
improved graduate, one more ready to contribute to his or her ship’s damage control 
readiness. 

The study was conducted using students from three RPL school classes that 
convened in March, April and May 1997. For this research, each student involved was 
given the option of not participating in the study. Of the 73 students attending the school 
during the three-month test period, all opted to participate in the experiment. The March 
and April classes were randomly divided into two groups (control and treatment), with 
the treatment group having their training augmented by use of the IDCTT (RPL). In 
April, however, the treatment group was comprised of volunteers. 

For the study, the control group proceeded through the school-prescribed training 
track. The treatment group also passed through the normal curriculum, but was also 
given standard scenarios during the course requiring the use of the IDCTT. During week 
two of the course, each student sat before an oral board (examination). These oral boards 
were conducted by two retired senior Naval officers, both with command experience. 
The “proctors” had no knowledge of the subject’s training track, ensuring an unbiased 
evaluation. Questions asked at this board were designed to determine level of knowledge 
based on standards established by the Repair Party Leader Personnel Qualification 
Standard. 

Data analysis revealed that the students that used IDCTT (RPL) as an augment to 
the established curriculum performed significantly better on the oral boards than did 
those in the standard training track. In fact, the treatment group posted a mean oral board 
score of 60.78, compared with 48.47 for the control group. These figures correspond to a 
quantitative learning advantage of 20.26%, a value that corresponds with results from 


similar studies regarding the effectiveness of interactive training systems. The May class 


Xi 


volunteers performed even better, showing a quantitative learning advantage of 77%. 
Based on the results, 1t can be concluded that use of IDCTT (RPL) at the Repair 
Party Leader School in San Diego did have a positive impact on the oral board scores 
recorded by the treatment group. This technology holds great promise for improving the 
training available to shipboard personnel. Improved training will contribute to enhanced 
damage control readiness, and will enable ships to meet the requirements of total ship 


survivability. 
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L INTRODUCTION 


Over the past several years, computer-based, interactive training has become an 
important part of the military’s education system. This technology, once out of reach due 
to cost, is gaining acceptance within the Department of the Navy as a means of either 
supplementing or replacing school-based. With the decreasing cost of Windows-based 
personal computers, the Navy is aggressively pursuing development and procurement of 
interactive courseware that will, by providing improved, “just in time” training, enhance 
the overall combat readiness of ships and aircraft within the force. 

Systems that have been developed for use by the Department of the Navy include 
the Interactive Damage Control Training Technology (IDCTT) for Damage Control 
Assistants, as well as interactive multimedia courseware used to provide challenging 
decision-making scenarios to junior officers in command of the CYCLONE-class PC’s 
(coastal patrol boats). In the instance of IDCTT, the primary goal of the software is to 
provide a supplement to training received in the traditional schoolhouse. On the other 
hand, the system employed for the commanding officers of the PC’s is used to expose the 
individual to “the worst day at sea”, causing him to have to make quick and accurate 
decisions in order to avert catastrophe and complete an assigned mission. (J. Miller, 
personal communication, September 26, 1996) 

The purpose of this thesis is to determine the effectiveness of the use of an 
interactive multimedia system in the training of Repair Party Leader (RPL) candidates in 
order to prepare them to assume the responsibilities associated with this position in the 
ship’s damage control organization. This research will examine whether the use of 
interactive courseware to supplement training received by students at the Repair Party 
Leader course conducted at U.S. Navy Fleet Training Centers will result in a graduate that 
is closer to achieving qualification as a Repair Party Leader. It is predicted that a student 
that has used IDCTT (RPL) in conjunction with the Repair Party Leader Course will have 
a significantly higher exit level of knowledge than one trained by traditional schoolhouse 


methods alone. 





IL. BACKGROUND OF INTERACTIVE LEARNING 


A. CONCEPTS OF INTERACTIVE MULTIMEDIA USED IN EDUCATION 
AND TRAINING 


Computer-based instruction (CBI) using interactive multimedia has gained 
increased acceptance over the last 20 years as an effective and inexpensive means of 
providing training to military personnel. This method is well suited to military 
applications as training resources dwindle and the requirement for personnel able to 
operate and maintain sophisticated systems increases. As a computer-based system, 
IDCTT (RPL) serves as a tool with which to assist the Repair Party Leader in developing 
decision-making skills based on a foundation of knowledge in basic and advanced 
damage control. 

Spiro (as cited in Seidel and Weddle, 1987) describes an “Experience- 
Consolidation System” as a computer-based teaching tool that assists the user in 
developing judgement and intuition that will result in better decision making skills. 
IDCTT (RPL) can be categorized as such a system in that it focuses on the user’s ability 
to make correct decisions and instigate action in a time-compressed situation. 
Experience-Consolidation Systems teach what is normally associated with repetition and 
practice in the field by helping to establish patterns for problem solving, and causing the 
user to “think for himself.” (Spiro, 1987) 

Taylor (as cited in Alessi & Trollip, 1991) classified instructional computing into 
three categories: tutor, tool and tutee. The tool and tutee categories do not apply to 
IDCTT (RPL), and are not discussed in this research. For computer-based instruction 
like that presented by IDCTT (RPL), the system serves primarily as a sort of tutor. 
Although it does not walk the trainee through a scenario, IDCTT (RPL) provides 
information and assessment via end of session feedback. Additionally, during the 
conduct of the scenario, the user has access to several options that serve as memory aids, 
or prompts, that will cause the user to draw on previously obtained knowledge of damage 


control theory, procedures and doctrine in order to make the correct decision. 


A difficulty with any computer-based instruction is that for the system to provide any 
positive result, the user must be comfortable with the operation of a personal computer 
(Alessi and Trollip, 1991). In today’s military, it is likely that more recently-inducted 
members will have frequently used computers prior to joming, where as personnel that 
have been in the service for several years may never have been exposed to them. This 
disparity may result in differences in results obtained by users of the computer-based 
learning system, and may discourage those that have little or no computer literacy. 

In order to maximize results, the computer should be used in training situations 
where it is likely to be beneficial (Alessi and Trollip, 1991). Providing training to 
candidates or serving Repair Party Leaders is a good application as the traditional 
training cost is high (ie. the individual Sailor is absent from his ship for 12 days), and the 
nature of the qualification process requires many repetitions of drills.) IDCTT (RPL) can 
provide low-cost training to individual students, reducing the impact this training has on 
the remainder of ship’s company. 

Alessi and Trollip (1991) describe instruction as a four-part process. The initial 
phase is that of “presenting the information.” In the case of the Repair Party Leader, this 
is accomplished through the trainee’s completion of prerequisite Personnel Qualification 
Standards and attainment of appropriate qualifications. Phase two of the process 1s that 
of “guiding the student.” IDCTT (RPL) provides a modicum of guidance via “pull- 
down” menu options that can be selected by the user, as well as prompts provided by the 
“phone-talker” and messages from the On-Scene Leader and Damage Control Central. 
The third and fourth segments of the process involve “practicing the student” and 
“assessing learning.” IDCTT (RPL) uses the concept of drills to enhance student 
performance, but, in the strictest sense, does not provide training for the user. Alessi and 
Trollip (1991, p. 108) stress that single drill sessions should not exceed 15 minutes in 
order to prevent user boredom or fatigue. As currently configured, IDCTT (RPL) dnl 
scenarios require between six and 15 minutes to complete. | 

Computer-based drills have the ability to provide a flexible, mteresting learning 


environment to the user. The employment of sound and video clips, as well as 


sophisticated visual graphics, provides the student with a realistic presentation of 
situations and any impact his or her decisions may have on the progress of the scenario. 
This multimedia environment may stimulate further study into the concept being pursued, 


and, as a result, a higher level of user ability. 


B. APPLICATIONS OF COMPUTER-BASED, INTERACTIVE AND 
MULTIMEDIA INSTRUCTION 


As mentioned earlier, the Navy is in the process of fielding several computer- 
based training systems. The U.S. Army has also invested resources in the development 
of interactive training systems in an effort to more efficiently spend scarce training 
dollars. One such system developed by the U.S. Army Signal Center at Fort Gordon, 
Georgia, involves the use of interactive videodisc technology to train communications- 
electronics military occupational specialties. This system uses a microcomputer and laser 
videodisc to provide an interactive learning environment for soldiers in these fields 
(Winkler and Polich, 1990). 

Another application of interactive training employed within the Navy involves the 
training of U.S. Marines in field first aid. (Dr. Eric Allely, personal communication, 
January 15, 1997) It is desirable that when in combat, every Marine be capable of 
performing life-saving measures on a wounded comrade, providing support until other 
field medical personnel arrive to treat the casualty. A system similar in design to IDCTT 
has been developed for this purpose, and field-testing indicates that it is an effective way 


to train Marines in this very valuable skill. 


C. PREVIOUS FINDINGS 


Interactive learning attempts to reduce training time required to raise an 
individual to a certain level of task proficiency. Studies conducted by Orlansky and 
String in 1979, and Orlansky, String and Chatelier in 1982 (as cited by Funaro and 
Lane in Seidel and Weddle, 1987) showed that in typical settings, computer-based 


learning applications “achieved an effectiveness equal or slightly superior to conventional 
methods with a time savings on the order of 30 percent.”’ While this research does not 
address the issue of time, the use of IDCTT (RPL) as a means of reducing time required 
to earn qualification as a Repair Party Leader is an area that provides opportunity for 
further study. 

Unfortunately, previous studies do not indicate that computer-based instruction 
provides a noticeable improvement when compared with training conducted using 
conventional methods, except in the application of reducing time required to learn a task. 
In fact, many studies indicate that there is no significant difference between results 
achieved by interactive and conventional instruction. Clark and Craig (as cited in 
McKenna, 1995) found that the use of multimedia technology is not itself the factor that 
influences learning, and that “measured gains in studies of the instructional uses of 
multiple media are most likely due to instructional methods such as interactivity that can 
be used with a variety of single and multiple media.” Alessi and Trollip (1991) report 
that “research aimed at proving that computer-based mstruction is better than using a 
book, teacher, or some other traditional method has shown only a small effect in favor of 
computer-based instruction.” 

Research conducted at the Naval Postgraduate School on a system similar to 
IDCTT (RPL) indicated that students were receptive to the training and felt as if they had 
learned more. In particular, Johnson (1994) stated that, as found by a study of the IDCTT 
(Damage Control Assistant) system, that “the IDCTT trainer is a highly effective training 
aid in a shore based environment.” In the same study, Johnson (1994) reports that when 
comparing actual performance scores obtained by students using traditional training 
methods with those obtained using IDCTT, ‘the IDCTT is clearly the more effective 


training medium.” 
D. IDCTT (RPL) SYSTEM DESCRIPTION 


The IDCTT (RPL) is a U.S. Navy-developed software program that provides 
realistic integrated shipboard casualty control training (“IDCTT Repair Party Leader”, 


N86D pamphlet). This system utilizes a large database and numerous video clips to 
provide the trainee with a very realistic training environment. In the system’s current 
configuration, the user is presented with an opening screen that allows him or her to 
select from a few scenarios, including damage resulting from a collision while underway 
and damage resulting from a collision while at anchor in a busy strait. All scenarios are 
designed around the characteristics and capabilities of the ARLEIGH BURKE-class 
guided missile destroyer (DDG-51). Figure 1 provides a sample view that would be 
presented to the user. In this view, the user (i.e. the Repair Party Leader) is able to 
observe messages sent by his investigators, as seen in the lower left section of the 
display. In Figure 1, the investigator has reported to Repair 3 that at time 1607, flooding 
water is four feet deep in compartment 4-370-5-A, Supply Storeroom Number 3. All 
messages sent and received during the IDCTT scenario are generated in the format 
prescribed by NWP-3-20.31, Surface Ship Survivability, ensuring continuity of training. 
Each time a message is received at the Repair Locker, the “New Report” pushbutton 
above the window flashes red, prompting the user to read the new message. By using the 


arrow button on the lower part of this window, the user can page through all messages, 
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Figure 1. Sample IDCTT Display (from OPNAV N86D pamphlet). 


reviewing status and progress of damage, as well as subsequent efforts to control the 
effects ofthe damage. The video clips are presented in the upper left-hand portion of the 
window. In the case presented in Figure 1, the Repair Party “phone talker” is (verbally) 
reporting the content of the message presented in the window. The “phone talker” will 
not report the substance of the message to the RPL until the “New Report” pushbutton is 
depressed, but will inform the RPL that he is in receipt of a new report. The longer the 
RPL waits to ask for the report, the more insistent the “phone talker” will become. This 
mimics the real life flow of information from the On-Scene Leader (OSL) to the Repair 
Party Leader, providing for enhanced realism and increased pressure for the RPL to act. 

In the right-hand section of the display, the user is generating a message to the 
On-Scene Leader in response to the report of flooding recetved at 1607. In this message, 
the Repair Party Leader is directing the OSL to establish flooding boundaries to control 
the effects of the damage and prevent progressive flooding. When the message is ready 
for transmission to the scene, the user will have identified primary and secondary 
flooding boundaries both forward and aft of the affected compartment, using the “pull- 
down” menu keys in the window. These “pull-down” menus provide a list of frame 
numbers from which the RPL may select to designate as a boundary.’ Selection of an 
incorrect or inappropriate frame will result in complications later in the scenario. When 
the message is complete, the user will transmit it using the “Send” action button at the 
bottom of the window. Actions previously directed by the Repair Party Leader are 
available for review by selecting the “Ordered Actions” pushbutton on the message 
dialogue window. 

Other features of the display include the stability/firemain and information 
sections. As can be seen in Figure 1, the stability/firemain section provides visual output 
of one of the two available inclinometers. Currently, the RPL has selected to view the 
heel inclinometer, indicating the list of the ship as a result of the damage situation in 


progress. He or she could select trim information by using the mouse to click on the 


' A frame is an athwartships strength member of a ship’s hull, numbered from the bow aft. Frame numbers 
are used as reference points to locate fittings and compartments (Noel and Beach, 1977). In the design of a 
ship, there are special frames at regular intervals that identify the location of primary watertight and 
smoketight boundaries. These particular frames are the ones normally selected by the RPL at which to 
establish fire or flooding boundaries. 


Trim action pushbutton, resulting in the display of another inclinometer.2 This 
information is important to the RPL as it provides a visual indication of the ship’s 
stability condition. The gauges on the right side of this window provide firemain 
pressure indications for the port and starboard loops of the DDG-51 firemain system. In 
the center of the display, the RPL has the option of selecting specific information 
regarding one of several topics. In this instance, the RPL is viewing the “Flood/Hull 
Damage” menu. From this menu, other actions are available including valve isolation, 
establishment of boundaries and dewatering options. Each menu presents a list of options 
for the RPL based on the general title of the section. When the user selects a subtopic, a 
window “pops up,” allowing the RPL the option of directing the OSL to take some 
action. The flooding boundary establishment window displayed in Figure 1 provides an 
example of what the RPL will see when a subtopic is selected. If he were interested in 
undertaking dewatering of the affected space, the RPL would be, for example, offered 
various pump options along with several other options. 

Also available to the Repair Party Leader via the IDCTT are damage control 
templates, organization charts, communications information and ship’s characteristics. 
Each of these items are required components of the damage control information library 
maintained in the Repair Locker. The damage control templates, for example, provide 
the RPL with the location of structural fire and flooding boundaries as designed in the 
ship’s construction. This information allows the RPL to make rapid and accurate 
decisions regarding the placement of personnel and equipment for controlling the spread 
of fire and flooding. Additionally, information is readily available regarding the location 
and content of such critical spaces as weapons magazines, hazardous materials storage 
facilities, and flammable liquid storerooms, as well as magazine sprinkler systems, Halon 


protected spaces, and other installed firefighting assets. 


* List is defined as the inclination of a ship about a fore and aft axis, while trim is the fore and aft 
inclination of the ship around an athwartships axis (Noel and Beach, 1977). 

* Halon is a firefighting agent that can be employed from a remote location. This system is generally 
installed in engineering, spaces, flammable liquid storerooms, and paint mixing and issue rooms. 
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E. PRIMARY GOAL OF IDCTT (RPL) 


The intent of IDCTT (RPL) is to provide the student with practice in collecting 
and processing information, coordinating responses, and maintaining the overall damage 
control picture. These skills are critical to the performance of the Repair Party Leader as 
an individual, and development of these abilities will enhance the overall combat 
readiness of the unit. The skills mentioned above are difficult to learn outside of actually 
imposing damage control scenarios and then evaluating the Repair Party Leader’s actions 
and responses. IDCTT provides the RPL candidate with unlimited opportunities to “test 
his or her mettle” against the computer, practicing personnel and equipment management, 
prioritization of effort and event sequencing. Feedback at the end of each scenario allows 
the RPL to determine his or her progress, identify areas of weakness, and focus on 
concepts requiring remediation. Additionally, the ship’s Damage Control Training Team 
will have access to these training reports, enabling them to pinpoint areas of weakness 


that can be corrected via tailored training sessions. 
F, FEATURES AND BENEFITS OF IDCTT (RPL) 


IDCTT (RPL) has several embedded features. First, the system offers multiple 
scenarios including flooding, stability, fire, structural damage, pipe ruptures and 
personnel casualties. This software also provides the support necessary to conduct 
simultaneous interactive team training. The ability to modify scenarios at the local 
command level enables the ship’s damage control training team to tailor training to 
specifically target areas of weakness identified during drills or other training sessions. 
IDCTT (RPL) provides immediate student performance evaluation and feedback, 
reinforcing the training received by the user. Finally, cross-platform connectivity 
between Windows-, Unix- and OS2-based operating systems allows for system 
flexibility. (Chief of Naval Operations (OPNAV N86D)). 

Numerous potential benefits are available through use of IDCTT (RPL). 
Currently, the system supports training requirements including PQS and Fleet Exercises 


and promotes standardization of fleet trammg. IDCTT (RPL) improves decision- 
making, teamwork and communications skills, reduces instructor and team-training 
workload, and supports flexible training schedules. The system will promote cross- 
training of key watchstanders by making available an educational tool that is fun and easy 
to use. Since the system allows for student performance evaluation and tracking, the 
qualification process can be standardized throughout the fleet, resulting in a greater 
transfer of knowledge from platform to platform. | 

One of the great difficulties facing the leadership aboard ship is ensuring the 
adequate training and cross-training of key members of the organic damage control 
organization.’ This newly-devised system will allow individual members of the team to 
pursue, either with or without supervision, knowledge of ship’s systems, proper event 
sequencing, intelligent application of assets based on priority and other important aspects 
of damage control without requiring the participation of every member of ship’s 
company. This is a definite improvement over the current methods required for training, 
and, in the era of reduced shipboard manning, will result in enhanced readiness aboard 
the ship. 


G. IDCTT (RPL) TEACHING POINTS 


The IDCTT (RPL) software is designed to address doctrme, techniques, and 
procedures provided in various Navy damage control publications including the 


following: 


* Surface Ship Survivability (NWP 3-20.31) 
° Practical Damage Control (Naval Ships Technical Manual, Chapter 079) 


® Repair Party Manual (COMNAVSURFLANTINST 3541.1C / 
COMNAVSURFPACINST 3541.4B) 


* Cross training refers to the activity of preparing each member of the Repair Locker to be able to assume 
more than one role in the locker. This redundancy in training results in the Repair Locker being able to 
sustain casualties, and continue to execute it’s damage control function without loss of continuity. 
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» Firefighting Ship (Naval Ships Technical Manual, Chapter 555, V1) 


. Repair Party Leader Personnel Qualification Standard (NAVEDTRA 
43119-G, Watchstation 318). 


This coverage has been accomplished by establishment and incorporation of 
teaching points into each scenario developed for IDCTT (RPL). A complete list of 
teaching points is provided as Appendix A. As an example, one of the teaching points 
addressed in a fire scenario is the concept of establishing fire boundaries. The applicable 
teaching point states that “any physical barrier can be a fire boundary. Ideally, fire 
boundaries are the bulkheads, deck, and overhead surrounding the fire. Secondary 
boundaries are generally set at watertight subdivisions or airtight boundaries.” The 
reference for this point is provided as Surface Ship Survivability (NWP 3-20.31). In this 
example, failure of the RPL to order fire boundaries, or order inappropriate fire 
boundaries will result in scenario events cascading to a predetermined “kill point,” a stop 
in the software sequence from which the RPL cannot recover, and the ship is lost due to 
catastrophic damage. 

Once the scenario is completed, the student is provided with a brief verbal and 
visual critique of his or her performance during the drill. This evaluation provides 
information on incorrect procedures, steps taken out of sequence, and other areas of 
difficulty. The assessment also provides a list of references that the trainee may review 


prior to undertaking a follow-on scenario. 
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1. THE REPAIR PARTY LEADER 


Effective damage control is clearly an important factor in the survivability of a 
ship at sea. This effort plays an important role in the ability of the ship to sustain 
damage, reduce the impact of the damage on vital combat systems and maintain the 
ship’s ability to continue its primary mission. An important aspect of how damage 
control contributes to surface ship survivability is the requirement to “improve the 
capabilities and skills of personnel to enable them to expeditiously handle recovery from 
combat scenarios or accidents and to promptly restore vital ship’s systems.” (NWP 3- 
20.31, paragraph 1.1) 

The effective control of damage in a shipboard environment requires the 
participation of personnel from every echelon of the ship’s manning structure. A typical 
shipboard damage control organization is as presented in Figure 2. As the senior person 


in the locker, the Repair Party Leader is in a position requiring expertise and knowledge 
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Figure 2. Shipboard Damage Control! Organization (Typical) (from NWP 3-20.31, p. 2-12). 


in the leadership and management of personnel assets, as well as a thorough knowledge 
of damage control theory, systems and fundamentals. Specifically, as stated in Surface 
Ship Survivability (NWP 3-20.31), the duties and responsibilities of the Repair Party 


include: 

« Maintenance of stability and buoyancy. 

° Maintenance of the ship’s structural integrity and maneuverability. 

9 Making repairs to electrical and sound-powered telephone circuits. 

2 Administermg first aid and transporting injured personnel to battle 
dressing stations without seriously reducing the DC capabilities of the 
Repair Party. 

e Being familiar with the assigned area and related DC systems and 
equipment. 


There are additional functions that lie within the purview of specific repair 
parties, but they will not be addressed in this thesis. 


A. REPAIR PARTY LEADER TRAINING PROGRAM 


iG Current Practice 

Today, training for candidate Repair Party Leaders occurs at Fleet Traming 
Centers in San Diego, California and Norfolk, Virginia. The course, Damage Control 
Repair Party Leader (K-495-0040), involves 10 days of instruction and covers a broad 
range of topics. The course mission is to “train personnel in pay grades E-5 and above in 
advanced damage control theory and techniques so that they may fill Repair Party Leader 
billets in the shipboard damage control organization, and to facilitate the proper 
management of repair party personnel in casualty situations under all shipboard readiness 


conditions.” (Fleet Training Center, 1993) 


The Repair Party Leader course is divided into two phases, each of which is five 
days in length. Both phases involve classroom lecture, written “practicals” and hands-on 
training evolutions. Phase II (second week) of the course focuses on aspects of 
Chemical, Biological and Radiological Defense. This thesis will study the impact IDCTT 
(RPL) has on student knowledge gained in Phase I. 


2 Repair Party Leader School Instructors 

Course instructors are highly qualified senior enlisted personnel (Chief Petty 
Officer or above) with vast experience in shipboard damage control. While on sea duty, 
each instructor has earned the Enlisted Surface Warfare Specialist designation, indicating 
the level of expertise that that individual brings to the job. Prior to becoming an 
instructor, the candidate must successfully complete the Instructor Training course. This 
course is the Navy’s primer in presenting educational techniques to the instructor prior to 
them interfacing with the student. Additionally, Fleet Trainmg Center San Diego Repair 
Party Leader instructors must successfully complete a local certification program prior to 


teaching a class. 


3. Instruction at the Repair Party Leader School 

Instruction at the school is presented primarily in the form of lecture and 
equipment demonstration. Class size is limited to an enrollment of 36 students, but the 
average class has approximately 25 students in attendance. During the period of 
instruction, students are exposed to information addressmg Damage Control 
Organization, Communications, Repair Locker Coordination, Biological and Chemical 
Warfare Defense, Radiological Defense and Radiological Plotting. As part of the 
training, students are required to complete several written practical exercises, as well as 
spend one day in a damage control simulator (“wet trainer’) and one period in a 
“confidence chamber” where they are taught the proper use and capabilities of the Navy’s 
gas mask. The first week of training, Phase I, provides in-depth exposure to various areas 


of damage control, including fire fighting, flooding effects and stability. Emphasis is 
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placed on developing base skills in: 


e repair locker administration and management 
e damage control communications 

e plotting and symbology 

e plugging, patching and shoring 


e other general DC topics 


A student meets graduation requirements by scoring 80% or higher on two written 
examinations. The first examination, given at the end of week one, tests the student’s 
knowledge of general damage control concepts and procedures, while the second 
examination covers aspects of Chemical, Biological and Radiological (CBR) Defense. 

By the limited nature of the course, as well as restricted facilities, the student has 
a very limited opportunity to actually coordinate the activities of a Repair Locker, 
manage personnel and equipment, or exercise decision-making techniques. It is thought 
that use of IDCTT (RPL) as an educational augment at the school will enhance this 
opportunity, and result in an improved graduate, one more ready to contribute to his or 


her ship’s damage control readiness. 


4. Written Examinations Administered at Repair Party Leader School 

The written examination is one of the main evaluative mechanisms used in the 
RPL course to determine a student’s exit level of knowledge. The test is an exercise in 
memorization of general damage control idiom and discrete activities (e.g. pipe patching 
or construction of shoring), and does not provide any indication of the student’s level of 
preparedness to actually lead a Repair Locker through an active damage control scenario. 
This method of assessment works well in determining whether the student meets the 
course graduation requirements. However, it is a common misperception in the fleet that 


RPL school graduates have spent a good deal of time actually managing and directing the 
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efforts of a Repair Locker while at the school. This leads the ship’s senior damage 
control leadership to overestimate the ability the graduate possesses when he or she 
returns to the ship, and creates frustration for both the RPL candidate and supervisory 
personnel while the trainee is pursuing final qualification as Repair Party Leader. 


B. PERSONNEL QUALIFICATION STANDARDS 


For many years, the Navy has relied upon Personnel Qualification Standards 
(PQS) to provide a “map” for a candidate working towards completing a qualification for 
a specific watch station. These standards provide the Sailor with the minimum 
requirements he or she must meet in order to become qualified in the watch station in 
question. The PQS will indicate prerequisite qualifications that must be earned in order 
to begin the next program, and decomposes the qualification process into Fundamentals, 
Systems and Watch Station tasks. 

NAVEDTRA 43119-G (Chief of Naval Education and Training) provides the 
applicable standard to which all Repair Party Leaders must comply in order to become 
qualified in that billet. Prior to commencing the qualification, the Sailor must have 
completed the prerequisites, which include the On-Scene Leader watchstation and 
fundamentals of Stability. Of course, the OSL PQS also has prerequisites, so the 
qualifications build on one another. From this PQS, it is clear that the focus of the 
training is based on coordination of Repair Party efforts in combating the various types of 
damage that can occur in a ship. Based on a comparison of the RPL Course syllabus and 
RPL PQS, it would appear that the school does not provide the hands-on opportunity that 
the candidate requires in order to complete the qualification, leaving that aspect of the 
training to the shipboard damage control training organization. This burden can be 
greatly reduced through use of IDCTT (RPL) as a vehicle to allow the RPL candidate to 
train individually and complete the various PQS requirements. As stated earlier, the 
damage control leadership reviewing trainee performance reports can maintain quality 


control by generating tailored training to address areas of weakness. 





IV. METHODOLOGY 


For this research, each student involved was given the option of not participating 
in the study. Of the 73 students attending the school during the three-month test period, 
all opted to participate in the experiment. Prior to taking the pre-test on the first day of 
the course, the students received a standard briefing regarding the purpose and goal of the 
study. 


This pre-test is given in support of research that is being done regarding 
the possible incorporation of interactive courseware in the training process 
at this school. Your participation in this study is purely voluntary, but you 
may have fun and really learn something! This test is given in order to 
determine the baseline level of knowledge each student has upon enrolling 
in the course. Your score on this exam will have no impact on the 
remainder of the course. The questions for this test will be multiple 
choice, true-false, and fill-in. Circle the correct answer or fill in the blank 
as required. 


After taking the examination, students were given a brief overview and 
demonstration of the IDCTT (RPL) system which was installed at Fleet Traming Center, 


San Diego, for the duration of the experiment. 


A. EXPERIMENTAL DESIGN 


In order to determine the effectiveness of the IDCTT (RPL), an experiment using 
the Untreated Control Group Design with Pretest and Posttest as described by Cook and 
Campbell (1979) was used. 


1. Assignment of Students 
a. March and April Classes 
In this experiment, as in any other, the random assignment of students in 
the Repair Party Leader classes to either the control or treatment group was an 


important phase. Randomness in assignment of subjects to the two groups was critical in 
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ensuring the equivalence of the groups. Clearly, assignment of subjects to one or the 
other groups by means of a truly random method (ie. coin toss) would be optimal. 
However, in the case of this experiment and based on experience with damage control 
training in the fleet, it was decided that several specific variables were considered 
important enough to include them in the process of assigning students to groups (Winkler 
and Polich, 1990). The groups were designated as indicated below: 


° Control Group. This group consisted of members of the RPL School class 
that were not exposed to IDCTT (RPL) 


2 Treatment Group. This group was comprised of the remainder of the RPL 
School class, those who have their traiing augmented by use of the 
IDCTT (RPL) 


To assure the random distribution of students, several factors were considered prior to 


making assignment of a particular student to a group. 


* Sea duty experience 
e Special/advanced qualifications held (1e. ESWS) 
e Gender 


° Paygrade (officer and enlisted). 


b. May Class 

A different approach was taken with regard to assigning students to the 
control group for the May class. Volunteers to form the treatment group were solicited 
from the class in order to test the hypothesis that willing subjects would perform better 
than subjects selected at random. In its shipboard application, IDCTT (RPL) will be 
available to all members of ship’s company. These sailors will be able to use the system 
at their discretion in order to enhance their knowledge of ship’s systems and damage 
control theory, as well as improve individual decision-making ability. For these 
reasons, the aspect of volunteerism as a means of capturing the essence of motivation 
as a factor influencing learning was adopted for this class. While certainly not random, it 


was thought that this approach would provide some useful results. 
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Ze Entrance Data 

Entrance data (Appendix B) was obtained from each student. By collecting this 
information, it was possible to split the class in such a manner that approximately half the 
women assigned were placed in the treatment group, while the other half were designated 
to the control group. Similar divisions were made with respect to paygrade, special 
qualifications held, and sea duty experience. For obvious reasons, this information was 
not considered in the case of the May class. Details regarding the composite population, 
as well as demographics of the control and treatment groups are presented in section B.1 


of this chapter and Chapter V.A. 


3. Implementation of the Experiment 


With the exception of the assignment of students to the treatment group 
mentioned in section IV.A.1.b, the experiment was identically conducted for the three 
classes. Both groups were subjected to a written pre-test (Appendix B) in order to 
establish a baseline level of knowledge. This test was, in fact, an alternate version of the 
actual test administered by the school to determine whether the student had met the 
requirements for successful completion of Phase I of the course. The cover sheet of the 
examination also included questions that were used for subject identification and data 
gathering that would be used in the analysis of results. The control group then proceeded 
through the school-prescribed training track (Fleet Training Center, San Diego, 1993), 
completing the course with the normally given post-test. The treatment group also passed 
through the normal curriculum, but was also given standard scenarios during the course 
requiring the use of the IDCTT. The normal post-test was then administered, completing 
this group’s training. During week two of the course, each student sat before an oral 
board (examination), similar to boards used to qualify Engineering Officers of the Watch, 
Officers of the Deck and Command Duty Officers. These oral boards were conducted by 
two retired senior Naval officers, both with command experience. The “proctors” had no 
knowledge of the subject’s training track, ensuring an unbiased evaluation. Questions 
asked at this board, and provided as Appendix C, were designed to determine level of 
knowledge based on standards established by the Repair Party Leader Personnel 
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Qualification Standard (Chief of Naval Education and Training). This oral examination, 
coupled with the end-of-phase written examination, fulfilled the requirement of the 


experiment’s post-test. 


B. STUDY SAMPLE 


Each ship is required to provide a mid-grade to senior enlisted member or officer 
to serve as Repair Party Leader in each of the ship’s Repair Lockers. These billets are 
active only when the ship is at Condition I, otherwise known as General Quarters, the 
highest state of combat readiness. Commonly, there are three Repair Lockers in a 
cruiser-destroyer type vessel, indicating the requirement to have at least three members of 
ship’s company trained and qualified to fill the RPL billet. A requirement for serving as 
an RPL is satisfactory completion of the formal Repair Party Leader Course, as well as 
applicable Personnel Qualification Standards. This experiment was conducted using 
students attending the Repair Party Leader Course conducted at Fleet Tramimg Center, 
San Diego. Each of these Sailors are prospective Repair Party Leaders aboard their 
assigned U.S. Pacific Fleet-ships, and are representative of the type individual who would 
logically be exposed to the training offered by the mteractive RPL Traier. In order to 


attend this course, each student must possess the following basic prerequisites: 


® Paygrade E-5 or above 
2 Graduate of Advanced Shipboard Firefighting School 


» Completion of rudimentary General Damage Control Personnel 
Qualification Standards. 


The Repair Party Leader Course is generally convened once per month, although during 
some months, no class is held. Maximum class size is 36 students, but on the average, 
each class has approximately 25 students enrolled. This experiment was conducted using 
classes convened in March, April and May 1997. During this period, 73 students 
attended the class and participated in the study. 
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C. METRICS 


For this research, several measures were collected. These included the previously 
mentioned pre-test administered prior to any exposure to IDCTT (RPL) or classroom 
instruction, as well as the regularly scheduled post-course examination required by the 
school. Finally, the oral examination was conducted for students in both the control and 
treatment groups. Data collected included the percentage of correct answers (of 30 
questions) on the pre-test, percentage of correct answers on the post-test (of 50 questions) 
and performance by the student on the oral board. While the first two measures are clear, 
the oral board requires some explanation. 

Oral board questions were developed using information contained in the Navy’s 
damage control publications, PQS and based on the author’s experience at previous 
Repair Party Leader qualification boards. The questions were designed to determine 
whether the students in the treatment group were further along than the control group in 
gaining the knowledge required to complete the requirements of Repair Party Leader 
qualification. Proctors were provided with specific instructions on how to conduct the 
boards, as well as guidance to be provided verbatim to the student before the board began 
(Appendix C). The student was read the question and then allotted one minute in which 
to provide an answer. Student responses that matched correct answers for the questions 
were marked on the proctor’s sheet, and, when one minute had passed, the next question 
was asked regardless of the status of the previous question. Scores were calculated by 
determining the percentage of correct responses provided out of the 49 possible answers. 


Incorrect answers were not taken into account. 


1. Variables 
For this experiment, the dependent variable was the oral board score recorded by 


each partietpating Sailor. Independent variables include: 


. Training system (i.e. RPL School, or RPL School and IDCTT) to which 
the Sailor was exposed 


e Sea duty experience 
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« Gender 
® Level of education 


° Computer Aptitude Index generated from portions of the Armed Services 
Vocational Aptitude Battery (ASVAB) scores attained by the Sailor at the 
time of induction. 


Zs Computer Aptitude Index (CAT) 

In order to capture some idea of each participant’s computer literacy, ASVAB 
scores were obtained from the Defense Manpower Data Center in Monterey, California. 
As mentioned earlier, data was available for 52 of the 73 subjects of the research. In 
order to develop the CAI, aspects of all elements of the ASVAB were examined in order 
to determine and rate applicability as a measure of computer ability, problem solving and 
decision-making skills. An ASVAB test writer at DMDC provided insight as to the 
intent of each of these subtest areas, and helped in identifying the most logical set that 
would compose an index of computer ability (R. Hamilton, personal communication, 
June 19, 1997). Of these elements, three were determined to be possible indicators of 


these skills: 


' Paragraph Comprehension (PC) 
’ Arithmetic Reasoning (AR) 
2 Coding Speed (CS). 


Paragraph Comprehension requires the individual to use information to generate a 
decision. This function can be directly associated with the idea that IDCTT (RPL) 
provides the student with reports, status and assets available, and then requires the 
student to make decisions based on that information. While the ASVAB does not address 
damage control issues, the assessment of decision-making ability is generally similar. 
Arithmetic Reasoning evaluates logical problem solving. A parallel between this portion 
of the ASVAB and the requirements of IDCTT (RPL) exists much in the same way as 
mentioned for the PC subtest. Finally, the third element of the CAI, Coding Speed 
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measures the individual’s ability for abstract reasoning. IDCTT (RPL) required this 
ability as the user must be able to postulate the downstream impact his or her actions may 
have on the progress of the scenario in order to successfully save his ship. 

The DMDC database provided three versions of the ASVAB score for each of the 
52 subjects. These scores include the Raw Score, Standardized Subtest Score and the 
Navy Composite Score. The Standardized Subtest Score was selected for incorporation 
into the CAI as it accounts for differences between the various versions of the ASVAB 
administered at Military Entrance Processing Stations (MEPS). By design, the 
Standardized Subtest Score ranges between 20 and 80, with a mean of 50 and standard 
deviation of 10. (R. Hamilton, personal communication, June 19, 1997) 

Based on the information obtained from DMDC, the CAI was developed as 
Equation 1. The ASVAB does not provide a direct measure of an individual’s computer 


prowess. Therefore, this equation assigns equal weight to the three components because 
CAI =(PC + AR+CS)/3 (1) 


there is no way to determine which of the elements is the most important in describing or 


assessing an-mdividual’s computer ability. 
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V. DATA ANALYSIS AND RESULTS 


The nature of data collected and the design of the experiment suggested using 
three different approaches for analysis. First, the complete set of data, including all 73 
participants was examined. Second, data collected from the May 1997 Repair Party 
Leader class was analyzed. As will be shown, the difference between mean treatment 
and control group oral board scores for this class was significantly higher than that of the 
other two classes, creating interest about the data set. Finally, a censored data set, one 
taking into account information obtained from Defense Manpower Data Center, was 
studied. 


A. SAMPLE DEMOGRAPHICS 


A wide variety of skill levels, ratings and experience was represented in the 
Classes studied. Additionally, each class was comprised of officers, enlisted personnel, 
men, women, and members of various ethnic groups and races. In order to establish a 
profile for each participant, data regarding educational background, race, gender and 
performance on standardized aptitude examinations was collected from the Defense 
Manpower Data Center (DMDC) master files. Unfortunately, of the 73 participants, 
DMDC held records on only 52 persons. Of the 21 social security numbers that produced 
no matching record, 12 were associated with the treatment group while the remaining 
nine were part of the control group. Further, 18 of the 21 records that produced no match 
in the DMDC database were junior officers, Lieutenant or below. It is possible that some 
of these officers never took an aptitude test, or that they have not been in the Navy long 
enough for the data base to reflect any information. The three remaining unmatched 
personnel were a Chief Petty Officer, Chief Warrant Officer and a Second Class 
Boatswain’s Mate. The absence of information on these Sailors is inexplicable.: Level of 
education data, however, was obtained on all 73 participants. Table 1 provides 


demographic information regarding the overall sample population. 
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Demographics and Background Total Control Treatment 


Sample Size (N) 73 36 37 
Male (percent) 83.56 86.11 81.1] 
Pay grade (rank) Distribution (percent) 
E-5 10.96 8.33 13.51 
E-6 24.66 27.78 21.62 
E-7 through CWO4 24.66 30.56 18.92 
O-1 and O-2 34.24 30.56 37.84 
O-3 and above 5.48 2.77 8.11 
Educational and Aptitude Characteristics 
Previous Education Distribution (percent) 
Some college or more 42.46 38.89 45.95 
High School Diploma 38.36 38.89 37.83 
GED Certificate 4.11 2.78 5.41 
Less than High School Diploma 137 0.00 2.70 
Unknown 13.70 19.44 8.11 
Career Characteristics 
Average Sea Experience (months) 48.49 
Warfare Qualified (percent)! 42.47 


Table 1. Sample Demographics. 


Clearly, the sample was overwhelmingly male, with only 16.5% of the 
participants being female. This is not considered unusual, as the Navy has only recently 
opened the majority of shipboard billets to females. Interestingly, among the officers in 
the sample, 26% were female, and most were assigned to pre-commissioning units. Of 
this group of female officers, 83% were assigned to the treatment group. Since the 
Repair Party Leader billets are normally held by senior enlisted personnel and junior 
officers, it is not surprising that almost 59% of all participants in the study were between 
the ranks of Chief Petty Officer (E-7) and Lieutenant (junior grade). Finally, the average 
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sea experience of the sample group shows the fact that several of the junior officers had 
no previous shipboard experience, while some of the senior enlisted personnel had 
upwards of six years at sea. 

As mentioned previously, three separate Repair Party Leader classes participated 
in this study. Figure 3 shows the officer and enlisted composition of each of the three 
classes, as well as the composite group when considering all 73 study participants. As 
this graph indicates, the first two classes had smaller officer representation (30.3% and 
16.7% for the March and April classes) than did the May session (57.7%). This 
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Figure 3. Breakdown by rank for each RPL School class. 


unbalanced condition may influence results, and bears attention. Figure 4 provides 
information regarding the distribution of officer and enlisted personnel by class 
convening date as well as group assignment. Figure 4 shows that class 3, convened in 
May, has a much higher percentage of officer representation in the treatment group, 62%, 
as opposed to classes 1 and 2 (50% and 25% respectively). The effect of this distribution 
will be examined to determine impact on the overall result of the study. Additionally, 
54% of the May class treatment group was composed of officers, as opposed to 36% and 
8% of the March and April sessions. Considering these disparities may help explain 





' Some participants have earned qualification as a warfare specialist (Surface Warfare Officer or Enlisted 
Surface Warfare Specialist). Information regarding, qualification requirements for these designations is 
available in various Navy directives, including the Military Personnel Manual (MILPERSMAN). 
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Figure 4. Distribution of Officers and Enlisted (by class and group). 


results obtained. 

In order to utilize education level and aptitude examination information provided 
by DMDC, the data set was censored to include only those personnel for whom this 
information was available. While this reduced the sample size to 52, it is important to 
determine if there is any relationship between performance and education/aptitude. 
Therefore, two presentations will be made; the first neglecting education and aptitude, but 
considering the entire data set, and a second that considers these factors in the analysis. 

Table 2 provides information regarding the censored data set. As mentioned 
earlier, DMDC was able to provide information on 52 of the 73 study participants. The 
information in Table 2 is provided to enable analysis of the impact the subject’s level of 


education and aptitude as determined by AFQT/ASVAB tests administered upon 


induction into the Navy. 
Demographics and Background Total Control Treatment 
Sample Size (N) a2 2) 25 
Male (percent) 84.62 81.48 88.00 
Race Distribution (percent) 
White 69.24 74.07 64.00 
Black 15.38 14.82 16.00 
Other 15.38 11.11 20.00 
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Pay grade (rank) Distribution (percent) 
E-5 
E-6 
E-7 through CWO4 
O-1 and O-2 
OQ-3 and above 
Educational and Aptitude Characteristics 
Previous Education Distribution (percent)? 
Some college or more 
High School Diploma 
GED Certificate 
Less than High School Diploma 
Unknown 
AFQT Score (mean) 
AFQT Mental Category (percent) 
I (93-99 percentile) 
Il (65-92 percentile) 
IIIA (50-64 percentile) 
IIIB (31-49 percentile) 
IV (10-30 percentile) 
V_ (1-9 percentile) 


Table 2. Sample Population Demographics (Censored Data). 


Table 2 shows that the distribution between the control and treatment groups is 
fairly consistent across the spectrum of variables. The small percentage of subjects that 
have earned a college degree skews the reflected percentages while the raw numbers are 


relatively close. As would be expected, the vast majority of subjects fall into Mental 


* Data provided by DMDC indicated that of the 52 subjects, 6 had unknown AFQT scores. As such, these 
personnel were excluded from the calculation of education level, mean AFQT and Mental Category figures. 
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52.18 
0.00 
0.00 
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17.39 
43.48 
8.69 
26.09 
4.35 
0.00 


20.00 
32.00 
28.00 
16.00 

4.00 


21.74 
56.53 
8.69 
4.35 
8.69 
65.78 


13.04 
34.78 
30.43 
21.74 
0.00 
0.00 


Categories HIA and II. The presence of one Mental Category IV subject in the treatment 

group is somewhat dampened by that group’s larger percentage of Category I personnel. 
Figure 5 provides a summary of the censored data set. When considering this 

data, the officer population has been reduced such that representation in the March and 


May classes is fairly consistent (35% and 27%), while officers comprise only 5% of the 
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Figure 5. Officers and Enlisted (by class) for the Censored Data Set. 


April class. Figure 6 provides the distribution of officers and enlisted by class and group 
for the censored data set. With the officer population reduced from 29 to 11, one 
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Figure 6. Distribution of Officer and Enlisted (by class and group) for the 
Censored Data Set. 


would expect there to be an effect in the distribution of officers throughout the three 


classes. From this graph, it is clear that the distribution of enlisted personnel between the 
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control and treatment groups is consistent across all three classes. With respect to the 
officers, there are none represented in the April session’s treatment group, and only one 
in the May treatment group. Over the spectrum of classes, however, the officers are 


evenly divided between the control and treatment groups. 
B. ANALYSIS OF THE COMPLETE DATA SET 


Prior to beginning the experiment, it was expected that the subject’s rank (.e. 
whether officer or enlisted), level of education, sea experience, advanced qualifications, 
pre-test score and time actually spent using the IDCTT (RPL) would be important factors 
in determining how well the individual performed during the post-treatment oral board. 
With this in mind, data was collected for the 73 participants to address these variables. 
Appendix D provides the complete data set; names and social security numbers of the 


participants are not included because of Privacy Act restrictions. 


1. Variable Selection 

As an initial action, stepwise regression was employed in order to assist in the 
selection of variables. (Hamilton, 1992) This search strategy helped identify the 
variables that had the most significance in the prediction of a student’s performance 
during the Oral Board. The difficulties Hamilton (1992) discusses regarding this method 
were noted, but it was decided that stepwise regression provided the simplest means of 
determining the optimal variable combination. 

Appendix E provides the results of stepwise regression on nine mdependent 
variables using the “forward inclusion” method provided by S-Plus. In this method, the 
process starts with no independent variables in the model, and then first adds the variable 
that has the largest simple correlation with the dependent variable. At each subsequent 
step, the independent variable that provides the largest decrease in R’ is added to the 
model. (Hamilton, 1992) As shown in Appendix E, the variables were added in the 


following order: 
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Group 

Class 

Pre-Test Score 

Rank 

ESWS Qualification 
Education 

IDCTT (RPL) Exposure Time 
Sea Duty Experience 

Gender 


CE ey SS ae eae 


Figure 7 provides a graphical representation of the impact the addition of each variable 


has on the reduction in the residual sum of squares (numbers on the x-axis correspond to 


residual sum of squares 
15000 16000 


14000 


13000 


2 4 6 8 
size 


Figure 7. Variable Selection via Stepwise Regression. 


the independent variable listed above). As shown in Figure 7, the curve has a steep slope 
until the model contains five variables, and then flattens out, indicating that the addition 
of variables six through nine do not provide a significant decrease in R’. This result 
provides visual evidence that level of education, time spent working through scenarios 
using IDCTT (exposure time), sea duty experience or gender did not noticeably influence 


the subject’s performance on the Oral Board. 
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Zs Initial Data Analysis 

In order to examine the data, a tree-based model was developed. (“S-Plus Guide 
to Statistical and Mathematical Analysis”, 1995) This model was used to further screen 
variables as to their relative importance and summarizes the complex multivariate data 
set generated by the study. In this model, predicted values of the independent variable 
are made based on various partitions of the data. Logical expressions are displayed at 
each split, providing information regarding the branches that follow. As is the 
convention, the logical expression is evaluated as being true for branches to the left and 
false for nght branches. Figure 8 provides the tree-based model generated by the 


complete data set using all predictor variables. This model provides some interesting 
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Figure 8. Tree-based model (Complete data set). 


insight into the data. Six members of the treatment group spent less than 6.5 minutes 
using the interactive system. Based on the partitioning algorithm embedded in the tree- 
based model method, no significant differences exist between the control and treatment 
groups when exposure is less than 6.5 minutes. For all practical purposes then, the 
branches on the left side of the root (IDCTT exposure time less than 6.5 minutes) can be 


considered representative of the control group, while the right side is associated with the 


treatment group. One feature provided by the tree-based model is a visual indication of 
the importance of each variable. As can be seen in Figure 8, the longest vertical branch 
of the treatment group-side of the tree is associated with the parent split “class.” The 
longer the branch, the more important the model considers the variable. As for the 


control group, the most important variables are pre-test score, followed by rank. 


a. Control Group Analysis 

With respect to the control group, the first partition in the tree is based on 
the pre-test score recorded by the student (Figure 8). Students that earned a score in 
excess of 78.335 could be expected to answer 62.59% of the oral board questions 
correctly, regardless of rank. This figure is higher (in most cases, substantially so) than 
oral board scores earned by students not doing as well on the pre-test. Mid-grade enlisted 
students (paygrade E-5 and E-6, with respective rank indicators of 1 and 2)’ scoring 
between 66.665 and 78.335 could be expected to correctly answer 43.73% of questions 
asked at the oral board, while those posting pre-test scores less than 66.665 would 
correctly answer only 34.36%. Within the control group, the class in which the student 
was enrolled was a factor for students in paygrades E-7 through O-4 (rank indicators of 3 
through 5). For these students, those in the May RPL class (class 3) scored 44.02% on 
the oral board, while those in classes convened in March and April (classes 1 and 2, 
respectively) earned slightly higher scores. As shown in the fourth echelon, students in 
the first two classes scoring higher than 71.665 on the pre-test did significantly better on 
the oral board. Even though the algorithm continues to split the data, it is evident that sea 
time was not a factor producing great differences in expected oral board scores within the 


control group. 


b. Treatment Group Analysis 
As mentioned earlier, the right side of the tree in Figure 8 represents the 


model associated with the treatment group. While much simpler, this portion of the tree 


> Subjects in this study consisted of personnel of paygrade E-5 (mid-grade enlisted) through O-4 (midgrade 
officer). In order to simplify analysis, integers representing the paygrades were assigned as follows: 1 — 
E-5, 2 — E-6, 3 — E-7 through W-4, 4 — O-1 and O-2, 5 — O-3 and above. An indicator of 2.5, therefore, 
delineates the boundary between E-6 and E-7. 
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provides some interesting information regarding the differences in the three classes. In 
this case, the algorithm partitioned the data based initially on the classes, dividing the set 
into March and April classes, and the May class. Within the March and April classes, sea 
time was an important factor in predicting performance on the end-of-course oral board. 
Those personnel with less than 44.5 months of sea duty could be expected to correctly 
answer 46.71% of the oral board questions, while those with more time at sea would be 
expected to score 7.26% higher. It is interesting to note that sea time was considered to 
be the eighth most important variable when stepwise regression was employed. For 
students in the May class, IDCTT (RPL) exposure time was the partitioning factor. In 
this case, those students with greater than 17.5 minutes of system contact time scored 
11.61% higher than did those with less time running damage control scenarios. The most 
interesting aspect of the treatment group-side of the tree is the range of expected oral 
board scores. These scores range from 46.71 to 85.42, a very wide span. This variation 
raises questions regarding data collected from the May RPL class, and provides support 


for the three-phase analysis. 


C. Combined Analysis 

As stated earlier in this chapter, the inconsistencies in oral board performance 
across the three classes is of interest. Since the tree presented in Figure 8 indicates a 
large difference in performance between the March and April classes and that of the May 
class, a second tree-based model was constructed. In this model (Figure 9), the left 
branches of the tree correspond to the control group, while the right side represents the 
treatment group. In this model, it is clear that the May class treatment group could be 
expected to earn substantially higher scores on the oral board than any other group, while 
control group oral board scores are roughly equivalent across the three classes. As the 
May class treatment group was composed of volunteers, this striking result may provide 
evidence supporting the belief held by many educators that “... motivation is the ultimate 


key to success in the classroom.” (Poirot, 1992) 
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3. Analysis of Variance 

As presented earlier, stepwise regression identified five significant independent 
variables. Based on that regression, ANOVA was conducted on a model constructed 
using these variables, with the oral board score as the response variable. Appendix F 
provides the results of this analysis, including interaction. A simple additive linear model 
was also constructed using the variables identified in the stepwise regression. As 


discussed by Hamilton (1992), the selection of variables using a search strategy such as 
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Figure 9. Tree-based model of Complete Data Set (Group only). 


as stepwise regression is not without pitfalls. Hamilton states that p-values generated 
using these variables are “unrealistically low,” casting doubt as to the meaning of the 
significance of each variable. With this in mind, Hamilton recommends using p-values 
for descriptive value only. Therefore, the following analysis must be accepted with 
caution. Table 3 shows that of the five variables, group and class are highly significant, 
rank and ESWS qualification are moderately significant, and pre-test score is 
insignificant. This simple model produced an R’ value of .3309, indicating that the 
model variables account for only 33% of the variance. |§ As shown in the results of 
ANOVA (Appendix F), of the five variables in the model, group, class and ESWS 


qualification are significant predictors of a student’s performance on the oral board, 
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while pre-test and rank are insignificant at a=0.05. The complex and uninterpretable 
interactions revealed by ANOVA suggested the adoption of a simple additive model 


using these three independent variables. 


Value Std. Error Tvalue  Pr(>It|) 
Intercept 12.8303 11.4069 1.1248 0.2647 
Group 11.3005 3.2835 3.4416 0.0010 
Class 6.8078 1.9894 3.4221 0.0011 
PreTest 0.1665 0.1388 1.2000 0.2344 
Rank 2.9968 1.6182 1.8520 0.0684 
ESWS 7.2689 3.9908 1.8214 0.0730 


Table 3. Linear Model with Five Variables Model. 


Figure 10 is a boxplot comparison of control and treatment group oral board 


scores. From this plot, it would appear that the treatment group recorded significantly 
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Figure 10. Boxplot comparison of Oral Board Scores (Complete Data Set). 


higher scores on the post-course oral board. However, variability between the two 
groups is inconsistent. The median score for the treatment group, represented by the 


white horizontal line across the box, was 57.14, while the control group median score 
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was 48.98. While this is a large difference, it is unclear as to whether the difference was 
the result of the treatment, a combination of the interaction of the independent variables, 
or chance. In order to determine the source of the difference, boxplots were constructed 
comparing the control and treatment group oral board scores as influenced by the 
important independent variables identified by ANOVA (class and ESWS qualification). 
As Figure 11 shows, there was greater variability for those students previously ESWS 
qualified, and those students enrolled mn the May session (class 3). The overlap of the 
quartile boxes would indicate that there is no difference between oral board scores when 


these variables are considered. 
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Figure 11. Boxplots of Complete Data Set variability. 


4. Hypothesis Testing 
In order to determine the effectiveness of IDCTT (RPL) over conventional 
schoolhouse methods in the case of the complete data set, the following hypothesis was 


tested: 


H,: The mean oral board score recorded by the control and treatment group were 
equal. 
versus 


H,: The treatment group recorded a higher mean oral board score. 
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As a prelude to actually testing the null against the alternative hypothesis, the 
power of the statistical test was calculated (Appendix G). Power is defined as the 
probability of finding the null hypothesis to be wrong, given the decision rule and the true 
parameter value under the alternative hypothesis (Larsen and Marx, 1986). In other 
words, power is a measure of the sensitivity of the test to detect differences in the 
parameters being compared. In the case of this analysis, the parameter of interest is the 
mean of control and treatment group oral board scores. Higher power over a broad range 
of hypothesized differences between the means would be ideal; of course, common to all 
power functions is a reduced capability to detect differences as they approach zero. 

For this analysis, it had to be shown that the variance of control and treatment 
group oral board scores was equal. In order to do this, a two-sided F test was conducted. 


The following hypothesis was checked using this test: 


H,: The variances of the mean oral board scores recorded by the control and 
treatment group were equal. 
versus 


H,: The variances were not equal. 


With an F-statistic equal to 1.9855, a p-value of 0.045 was produced. Based on this 
result, and at a level of significance of 0.05, the null hypothesis would be rejected. This 
small margin of rejection, however, allows for the statement that the variances were not 
significantly different, and testing could continue. 

With equal variance and normality (essentially) shown, the test statistic generated 


(Equation 2) is distributed as a ¢ random variable with n+ m- 2 degrees of freedom 


eee VN 
ee Yo " 
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(where n and m are the sizes of the control and treatment group samples). When sample 
size is large, the ¢ distribution looks Normal. As such, the power functions developed are 
Normal approximations. 

Figure 12 is the power function developed using the complete data set. As would 
be expected, power is lowest when the control and treatment groups are hypothesized to 
be equal. In fact, the power at this poit is equal to the level of significance of the test, mn 
this case a = 0.05. What Figure 12 provides is an indication of the test’s ability to detect 
differences between the means over the range of —30 (i.e. the control group mean is 
larger) to 30 (i.e. treatment group mean is larger). As shown, this test has relatively high 
power and can be expected to detect a ten percent difference between the means with 


probability approximately 0.40. As the hypothesized difference becomes larger, the 
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Figure 12. Power Function for Complete Data Set. 


power of the test increases until it approaches one. So, for a large difference between the 
control and treatment group means, this test is likely to detect a difference. Conversely, 
for small differences (less than ten percent), the power of the test is limited. | 

Prior to testing this hypothesis, however, the data was explored to determine the 
nature of the underlying distribution. Figure 13 provides four plots used as diagnostic 
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tools with which to begin the analysis of the data. The top left plot is a histogram of the 
data. The lower left plot is known as a density plot. This plot is essentially a smooth 
version of the histogram and provides smooth estimates of population frequency or 
probability density curves. As they are similar, the density curve and histogram should 
have basically the same shape. However, differences in how the software package 
generates bin sizes can result in differently shaped plots. Based on the histogram in 
Figure 13, the data appeared to be positively skewed. By applying a power 
transformation (raised to the 0.8 power), the distribution of the data was made more 
symmetrical. The histogram and boxplot (Figure 14) indicate the effect of the 
transformation. Based on the visual evidence provided m Figure 14, the transformed 
data was assumed to be normally distributed, permitting the application of standard 


Statistical tests of the previously mentioned hypothesis, using the transformed data. 


Oral Board Score Oral Board Score 








(Complete Data) (Complete Data) 
dh. c 
_|a a . 
20 @ oo 80 100 
Oral Board Score Oral Board Score 
(Complete Data) (Complete Data) 
=e - 
0 yal) @ 60 ao 100 120 2 1 0 1 2 


Figure 13. Diagnostic Plots for Complete Data Set. 
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To test the hypothesis that the mean oral board scores recorded by the control and 


treatment group were equal, a two-sample t test was conducted on the transformed 
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Figure 14. Diagnostic Plots for Transformed Data (Complete Data Set). 


data. This test indicated that at a level of significance of 0.05 and a f-statistic of 3.4026, 
there was a difference between the means (p-value = 0.0006), and the null hypothesis was 
rejected. The treatment group, in fact, attained a mean oral board score 20.26% higher 
than that of the control group (60.78 to 48.47). It could not be measured, however, the 
exact contribution use of IDCTT (RPL) had in this learning benefit. 


S: Analysis of Results 

Since this experiment was based on the assertion that use of interactive 
courseware as an instructional aid benefits learning, the discovery that the group the 
student was part of (1.e. control or treatment) was significant is not surprising. As 
previously mentioned, the treatment group used volunteers from the May class, so the 
oral board performance differences between the classes could be expected. Table 4 
provides a principal component analysis of the independent variables in order to further 


explain the factors contributing to differences in oral board scores. From this table, the 


class the student was in accounts for 64% of the variance in the model. However, 31% 


of the variance is explained by group affiliation. This information provides further 


Standard Deviation Proportion of Variance Cumulative Proportion 


Class ti“(<i‘étlédéww2z5«~0©6©€©™©™~™~C(687580300—=—=—C™t~é“‘«‘é«C OW 37SBO——Ct~«*é 

Group 27.88 0.3060058 0.9435861 
Rank 72 0.05407455 0.99766068 
Education 2.03 0.001618563 0.999279238 
ESWS 0.90 0.0003188014 0.9995980396 
Gender 0.79 0.0002443316 0.9998423712 
Sea Time 0.44 0.00007777285 0.99992014409 
Pre Test 0.33 0.00004408595 0.99996423004 
Exposure Time 

0.30 0.00003576996 1.00000000000 


(< 6.5 minutes) 
Table 4. Variance as explained by Principal Components. 


evidence that use of IDCTT (RPL) provides some educational benefit to the students in 
the class. However, the fact that IDCTT (RPL) exposure time was inconsequential 
creates concern as to the quantitative advantage gained by students in the treatment 
group. The importance of the class the student was enrolled in has to be considered as a 
confounding variable. Since this component accounted for a majority of the variance in 
the model, it must be addressed. Analysis of data collected from the May class will 
hopefully clarify this issue. 


OE ANALYSIS OF DATA COLLECTED DURING MAY 1997 RPL SCHOOL 
CLASS 


Techniques employed in examining the complete data set were used in exploring 
data collected from the May RPL School class (Appendix D). As mentioned earlier in 
this chapter, obvious differences in oral board performance among members of the 


treatment groups necessitated this separate analysis. 
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1. Variable Selection 
Stepwise regression was employed using the eight available variables. As 


illustrated in Figure 15, this regression identified six variables that made important 
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Figure 15. Variable Selection via Stepwise Regression. 


contributions to the reduction in the residual sum of squares. As such, the following 
variables were used in the construction of a simple additive linear model and conduct an 


ANOVA: 


L. Group 

2 Pretest Score 

3. Exposure Time 

4. Sea time 

> ESWS Qualification 
6. Education 


Ze. Distribution of the Data 
Figure 16 provides a series of diagnostic plots used to determine the 


characteristics of the underlying distribution of the data. The QQ-plot and boxplot 
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provide an indication that the data set has light tails and very few outlying points. The 


information provided in Figure 16 does not, however, provide much clarification as to the 
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Figure 16. Diagnostic plots for the May 97 Class. 


distribution of the data. To further examine the data, composite goodness-of-fit tests 
were applied. In these tests, the parameters required (mean and standard deviation) were 
estimated from the data. As a result of parameter estimation, the goodness-of-fit tests 
were limited in their ability to determine the distribution and specific parameters 
underlying the data. In fact, composite goodness-of-fit tests can only determine the 
family of distributions to which the data belongs (S-Plus, Guide to Statistical & 
Mathematical Analysis”, 1995). 

The null hypothesis for these tests was that the cdf of the data equaled the Normal 
distribution, with the alternative being that they were not equal. Figure 17 1s a plot of the 
empirical cdf and hypothesized Normal cdf: This plot provides a foundation that the data 
may be assumed to belong to the Normal family of distributions, a conjecture that was 
further examined by goodness-of-fit tests. The first goodness-of-fit test applied was the 
one-sample Kolmogorov-Smirmov test. Using the estimated parameters, the null 
hypothesis could not be rejected (p-value = 0.5) at a 5% level of significance. In order to 
confirm this result, the chi-square goodness-of-fit test was applied to the same 
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hypothesis. Again, the results of this test do not permit the rejection of the null 
hypothesis that the data is from a non-specific Normal distribution (p-value = 0.1649). In 


both cases, the p-values returned caused suspicion about the null hypothesis, but did not 





30 40 50 60 70 80 90 


Solid line is the empirical cf. 


Figure 17. Empirical and Hypothesized Normal CDF’s. 


allow for definite rejection at the level of significance specified. In particular, the chi- 
Square test p-value was suspect as the test was applied to a small sample, when it is most 
effective for large data sets (“S-Plus, Guide to Statistical & Mathematical Analysis”, 
1995). 

Based on the visual evidence presented in Figures 16 and 17, as well as the results 
provided by the goodness-of-fit tests, the data were assumed to be Normally distributed 


and were analyzed as such. 


3. Initial Data Analysis 

As mentioned earlier in this section, group, pre-test score, exposure time, sea 
time, ESWS qualification and education were found by stepwise regression to be the 
most important factors with regard to the May 97 Repair Party Leader School class 
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(Appendix E). A tree-based model, as discussed in Chapter IV.A.2, was constructed, 
using all six variables. As shown in Figure 18, the partitioning algorithm identified only 
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Figure 18. Tree-based model (May 97 class data set). 


two of the variables as being significant for analytical purposes. The first division was 
made based on IDCTT exposure time. In Figure 8 the partitioning algorithm split the 
data based on 6.5 minutes of exposure time. Using the May 97 class data, this same 
algorithm divided the data at five minutes of exposure. All 13 members of the treatment 
group in the May class had greater than five minutes of use of IDCTT (RPL). Thus, as in 
the previous tree (Figure 8), the left branch of Figure 18 represents the control group 
(exposure time < 5 minutes), while the right side corresponds to the treatment group. It 1s 
clear from Figure 18 that the May class treatment group performed at a much higher level 
on the oral board than did the control group. With regard to the control group, sea duty 
experience was a major factor in influencing oral board performance, with the division 
made at just over two years. In this particular class, the treatment group scored an 


average of 78% higher on the oral board than did the control group. 
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4. Analysis of Variance 

In order to determine the importance of the independent variables identified in the 
stepwise regression, ANOVA was conducted. Appendix J provides the complete result 
of this procedure, and includes all interaction effects. From this ANOVA, only group is 
significant at the 0.05 level, while pre-test score and ESWS qualification would be 
significant at a 10% level of significance. This result was expected, as the difference in 
oral board scores between the control and treatment groups was substantial. In order to 
provide for comparison with the results attained by ANOVA, a simple additive linear 
model was developed using the same independent variables. Table 5 shows that at a = 


0.05, group and sea duty experience were significant. At a 10% level of significance, 


Value Std. Error _t value Pr(>|t|) 

‘Intercept 35.9033 13.4969 = 2.6601 ~—s (0.0155 _— 
Group 21.6808 6.0395 3.5898 0.0020 
PreTest 0.1445 0.1732 0.8340 0.4146 
ExpTime 0.4137 0.2088 1.9809 0.0623 
SeaTime -0.2300 0.1006 -2.2857 0.0339 
ESWS 13.0991 6.6580 1.9674 0.0639 
Education 0.8156 0.8229 0.4208 


0.6712 
Table 5. Linear Model for Six Variables (May Data Set). 


IDCTT (RPL) exposure time and ESWS qualification were shown to be important factors 
in predicting oral board performance. The linear model generated in the case of the May 
Class data provided a very good fit and explained almost 89% of the variance in the data 
(R’ = 0.8865). 

Principal component analysis was conducted on this data set in order to identify 
the variables that explained the majority of the variance. As shown in Table 6, the group 
the student was in accounts for 78% of the variance in the model, while rank and 


education explain 15% and 7% respectively. Combined, these three variables explain 
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- Standard deviation Proportion of Variance | Cumulative Proportion — 
Group Bosca lou ae OUME3859m = =i 7.3859 
Rank 13.9610838 0.1465130 0.9238988 
Education 9.82459602 0.07255497 0.99645382 
ESWS 1.969600738 0.002916045 0.999369860 
Gender 0.7989040141 0.0004797632 0.9998496232 
SeaTime 0.32548378437 0.00007963362 0.99992925684 
PreTest 0.25071729527 0.00004725049 0.99997650733 
Exposure Time 0.17678575526 0.00002349267 1.00000000000 


Table 6. Variance as explained by Principal Components (May Class Data Set). 
99.6% of the variance in the data. Of interest, this was the only instance where 
education was a significant factor in the linear model. Eight of 13 students (61.5%) in the 
May RPL class treatment group had earned college degrees. This, coupled with the fact 
that they volunteered to be in the treatment group most likely influenced the oral board 


scores, and would explain the appearance of education as a significant variable. 


3s Hypothesis Testing 

In order to determine or quantify the impact on preparing a candidate Repair Party 
Leader gained by using IDCTT (RPL), the hypotheses generated in Chapter IV.A.4 were 
tested. As stated in Chapter IV.B.2, the dependent variable was assumed to be Normally 
distributed, enabling the use of parametric methods for hypothesis testmg. In order to 
satisfy the assumption of equal variance between control and treatment group oral board 


scores, the following hypothesis was evaluated, using a two-sided F-test: 


Ho: The variances of the May class oral board scores recorded by the control 
and treatment group were equal. 
versus 
Hi: The variances were not equal. 
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The F-test yielded a test statistic of 0.8531 and a p-value of 0.7876. Based on a level of 
significance of 0.05, the null hypothesis could not be rejected. Therefore, for the 
purposes of this analysis, the variances of the control and treatment group oral board 
scores were assumed to be equal. 

As discussed in Chapter V.A.4, a power function was developed for the May 97 
data set. Figure 19 indicates that, as was the case with the complete data set, the test of 


hypothesis has greater ability to detect larger differences between mean control and 
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Figure 19. Power Function for May 97 Data Set. 


treatment group oral board scores. This curve was developed using a range of 
hypothesized differences in control and treatment group means over the interval (-40, 40) 
due to the fact that the actual difference is 35.008. As Figure 19 shows, the power of the 
test to detect a difference between the means equal to 20 percent is approximately 0.80, 
while if the mean difference were 10 percent, the power of the test is reduced to 
approximately 25%. 

To test the hypothesis that the control and treatment group oral board mean scores 
were equivalent, a two-sample ¢ test was conducted. This test revealed that, at a = 0.05, 
the null hypothesis was rejected (p-value = 0). In order to determine the true magnitude 
of the difference between control and treatment group mean oral board scores, the 


hypothesis was tested using an arbitrary value for the difference in the means. Since the 
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calculated difference is approximately 35, the value for the test was established arbitrarily 
at 42.5. When the null hypothesis is changed to test whether the difference between the 
treatment and control mean oral board scores is greater than 42.5, the null hypothesis is 
rejected, but with a much higher p-value (0.0487). The true treatment group mean 
(80.06) is significantly larger than the true control group mean (45.05538), indicating that 
for the May class, the use of IDCTT (RPL) may have had a significant impact on oral 


board performance. 


6. Analysis of Results 

Clearly, the May class treatment group performed better on the oral boards when 
compared with any other group in the study. The fact that IDCTT (RPL) exposure time 
was significant at the 10% level (p-value = 0.0623) is interesting in that this class was the 
only one where use of the system reflected any significant benefit to the students in the 
treatment group. Within the treatment group, it would appear that exposure to more than 
three IDCTT (RPL) sessions was important in the student’s performance on the oral 
board. As shown in Figure 18, a student with more than 17 minutes of time using IDCTT 
would could be expected to score nearly 12 points higher on the oral board than one with 
less exposure. This data division is the only one tested that clearly indicates the benefit 
of increased system usage. Using volunteers as the treatment group may have 
compromised the validity of the experiment somewhat, but the results may provide a 
better indication of what can be expected in fleet application. The larger proportion of 
officers using the system in the May class also creates interesting questions regarding 


computer ability. 
D. ANALYSIS OF CENSORED DATA SET 

As mentioned in Chapter [V.B.1, additional variables were available for 52 
participants of the study. The censored data set is provided in Appendix D. Subject race, 


Armed Forces Qualifying Test Score and Mental Category, as well as scores achieved on 
the various Armed Services Vocational Aptitude Battery subtests were provided by the 
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DMDC database. As such, this information was included in the analysis in order to 
determine variable influence on each subject’s performance during the Oral Board 


administered at the conclusion of the Repair Party Leader course. 


13 Variable Selection 
Appendix E provides the results of stepwise regression on 13 independent 
variables using the “forward inclusion” method provided by S-Plus. As shown in 


Appendix E, the variables were added in the following order: 


Race 

Group 

ESWS 

Class 

Pre-Test Score 

Sea Duty Experience (SeaT1ime) 
Rank 

Mental Category (MenCat) 
AFQT 


SOE ee Te Og ee ee 


— 
oe 


Exposure Time 


pd 
— 


Education 


— 


Computer Aptitude Index 
Gender 


— 
= 


Figure 20 provides a graphical representation of the impact the addition of each variable 
has on the reduction in the residual sum of squares (numbers on the x-axis correspond to 
the independent variable listed above). As shown in Figure 20, the curve has a steep 
slope until the model contains six or seven variables, and then flattens out, indicating that 
the addition of variables eight through 13 do not provide a significant decrease in R’. 
This result provides visual evidence that level of education, computer aptitude, gender or 
time spent working through scenarios using IDCTT (exposure time) did not noticeably 


influence the subject’s performance on the Oral Board. 
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Figure 20. Variable Selection via Stepwise Regression. 


2 Analysis of Variance 

Using the results of the stepwise regression, ANOVA was conducted on a model 
employing the best six variables. Appendix F provides the results of analysis of variance 
of the six variable model, including interaction. As shown, this analysis indicated that 
race, group (1.e. control or treatment), ESWS qualification, class and pre-test score were 
significant predictors of a student's oral board score. Rank, while one of the variables 
identified during the stepwise regression aS important, is an insignificant predictor of 
performance on the oral board as indicated by this model (p-value = 0.1289). The 
complicated interactions present between the variables are difficult to understand or 
interpret; therefore, these interactions will be disregarded for this analysis. 

A linear model was developed using the results of the ANOVA to further explain 
the data. This model eliminates the contribution of rank, and renders the output presented 
in Table 7. This model provides an R’ of 0.4175, indicating that 42% of the variance in 
this data is explained by these six variables. Of note, the score obtained by the student on 
the pre-test is a poor predictor of how that student will do on the end of course oral board. 
Additionally, the class the student was in is insignificant when examined in this context. 
This result will be shown to be of questionable value in later discussion. The linear 
model also confirms that pre-test score is an insignificant predictor of oral board 


performance. 
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Value Std Error t value Pr(>|t|) 


Intercept — 30.9527 12.9658 2.3873 0.0211 
Race -7.6349 2.6019 -2.9344 0.0052 
Group 10.4085 3.8609 2.6959 0.0098 
ESWS 8.4458 3.8880 2.1723 0.0350 
Class 4.2186 2.4236 1.7406 0.0884 
PreTestScore 0.2577 0.1563 1.6491 0.1059 


Table 7. Reduced (five variable) Linear Model (Censored Data). 


Within-group variability was larger for the treatment group, suggesting there was 
an uneven distribution of achievement using IDCTT (RPL). Figure 21 shows that, while 
the median score attained by the treatment group was higher, this group also had slightly 
higher variability (o’ = 344.83 for the treatment group, o’ = 161.44 for the control group). 
The variance of the treatment and control group mean oral board scores are formally 


examined in section C.3 of this chapter. 


Oral Board Score 





Control Treatment 


Figure 21. Boxplot of Control and Treatment Group Oral Board Scores. 
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The uneven distribution of achievement for the treatment group could be 


attributed to any one or all of the variables. Figure 22 examines four of these variables: 
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Figure 22. Boxplots of Treatment Group Variability. 


ethnic group, advanced qualifications, class in which the subject was enrolled, and 
paygrade. These plots indicate that there was a visual difference in performance within 
the treatment group based on ethnic group, ESWS qualification and, most strikingly, the 
class the student attended. The evidence that class is significant contradicts earlier 
findings, but makes sense when the uncensored data is reviewed. Based on the boxplot in 
Figure 22, it does not appear very important whether the student was an officer or 
enlisted. These results generally support the findings of the previously conducted 
ANOVA. 


3. Hypothesis Testing 

The power function developed for the censored data set is presented as Figure 23. 
Since the actual difference between the control and treatment group mean oral board 
scores was 10.27314, the range of hypothesized differences was established at (-30, 30). 
In contrast to the previously shown power functions, the test in this case has power equal 
to approximately 0.65 when the hypothesized mean difference is equal to ten percent. 


This test is more powerful across the range of hypothesized differences, providing a 
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Figure 23. Power Function for Censored Data Set. 


power equal to 1 at A = 15 percent (where A is the difference between treatment and 
control group mean oral board scores), as opposed to values greater than 20 percent for 
the other data sets. 

The issue remains to determine if use of the IDCTT (RPL) system enhanced the 
performance of the students on the oral board. Figure 24 provides some evidence that the 


treatment is effective, but the extent of the benefit of the IDCTT (RPL) in enhancing oral 
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Figure 24. Diagnostic Plots for Censored Data Set. 
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board performance is unknown. The histogram and QQ-plot in Figure 24 indicate the 
near normality of the oral board scores recorded from the censored data set. The QQ-plot 
indicates, however, the presence of positive skew and the possibility of high outliers. 
Several transformations were attempted, but none produced results that appeared more 
normal, so the raw data was retained, and was assumed to be Normally distributed for 
analytical purposes. 

Once the normality of the data had been established, a test for the equality of 
sample variance was in order. To determine whether the variance of the control and 
treatment group oral board scores were the same, the F-test was used to check the 


following hypothesis: 


H,: — The variances of the censored data set oral board scores recorded by the 
control and treatment group were equal. 
versus 


H;: The variances were not equal. 


Under this procedure, an F-statistic of 2.1359 was produced, along with a p-value of 
0.0615. Based on a level of significance equal to 0.05, the null hypothesis could not be 
rejected. Therefore, for the purposes of this analysis, the variances in question were 
assumed to be equal. 

In order to determine the relative performance of the students in the treatment 


group relative to that of the control group, the following hypotheses were established: 


H.: | The mean oral board score recorded by the control and treatment group 
were equal. 
versus 


H;: The treatment group recorded a higher mean oral board score. 
To test this hypothesis, a two-sample ¢ test was conducted. This test indicated that at a 


level of significance of 0.05 and a /-statistic of 2.3434, there was a difference between the 


means (p-value = 0.0116) and the null hypothesis was rejected. In fact, the mean oral 


Se) 


board score of the control group was 48.83, while the treatment group recorded a mean of 
59.10. As a result of the use of the IDCTT (RPL), the treatment group achieved a 
learning benefit of 21.03% over the control group, a finding that is consistent with 
improvements noted in other studies of the effectiveness of multimedia training systems 


when used in military applications (Fletcher, 1991). 


4, Analysis of Results 

Initially, it was thought that level of education and computer aptitude would have 
a noticeable impact on a student’s ability to use the IDCTT (RPL), and, consequently on 
his or her performance during the post-treatment oral board. Clearly, this was not the 
case. In fact, education, and computer aptitude were eleventh and twelfth in importance 
as shown by stepwise regression. This result was counterintuitive. As one would expect, 
however, being in the treatment group was a significant factor contributing to the higher 
oral board scores, indicating that the students did receive benefit from exposure to the 
interactive multimedia learning system. The class the student was in was also significant, 
leading to the conclusion that either there were differences in the three learning groups 
(i.e. the volunteers in the May class performed better), or that the conduct of the oral 
board was inconsistent between classes convened in March and April and the class 
convened in May. Qualifications earned prior to attending the RPL course also had an 
effect on the result. A priori, it was thought that students who were Enlisted Surface 
Warfare Specialists or Surface Warfare-qualified officers would perform better on the 
oral board, regardless of the training track assigned. This was borne out by the results in 
that ESWS was a significant factor. 


E. SUMMARY OF RESULTS 


Table 8 provides summary information regarding the results obtained from this 


study, using the hypothesis listed below: 
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H,: The mean oral board score recorded by the control and treatment group were 


equal. 


versus 


H,: The treatment group recorded a higher mean oral board score. 


Data 


Complete Set 
May 97 Class 
Censored Set 


Significant Variables Result of Mean OBS Quantitative 
(ANOVA) Hypothesis Test ———————_—_-—--____ Learning 
Advantage 
Control Treatment 
Group, Class, Null rejected 48.47 60.78 20.26% 
Group Null rejected 45.06 80.06 77.54% 
Race, Group, ESWS, Null rejected 48.83 59.10 21.03% 
Class, PreTest 


Table 8. Results Summary. 
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VL CONCLUSIONS AND RECOMMENDATIONS 


Ae CONCLUSIONS 


The purpose of this thesis was to determine whether using IDCTT (RPL) as a 
supplemental learning tool would produce a Repair Party Leader School graduate that 
was Closer to qualification than those students trained using the traditional schoolhouse 
method alone. As stated in Chapter V.A.4, the following hypothesis was tested to 
determine the impact use of IDCTT (RPL) had on the learning of students in RPL 
School: 


H,: The mean oral board score recorded by the control and treatment group were 
equal. 
versus 


Hj: The treatment group recorded a higher mean oral board score. 


Based on the information contained in Table 8, it can be concluded that use of 
IDCTT (RPL) at the Repair Party Leader School in San Diego did have a positive impact 
on the oral board scores recorded by the treatment group. While the amount of advantage 
gained by the treatment group was fairly consistent between the complete data set and the 
members of the censored set, the huge advantage gained by the treatment group in the 
May class is noteworthy. Leaming advantage may not be a good measure of progress 
along the curve of final qualification, but performance on the oral board provides an 
indication as to whether the RPL candidate possesses the basic knowledge expected of a 
Repair Party Leader aboard ship. 

As previously mentioned the treatment group for the May class was comprised of 
volunteers. Review of literature presenting experiments of this type did not address the 
effect motivation would have on the results. Since the May class performed substantially 
better than the other two classes, it would appear that the motivated student is able to 


achieve greater benefit by using the system than a possibly resistant subject. Of course, it 
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could be hypothesized that the motivated student would perform better anyway, 
dampening the attribution of success to use of IDCTT (RPL). 

It was surprising that previously attained advanced qualifications (ESWS), rank 
and education (including computer aptitude) were not found to be significant factors in 
this research. This lack of significance would lead to the belief that IDCTT (RPL) is a 
very user-friendly system, one that makes it easy for the student to proceed through the 
scenarlo, gaining experience along the way. Additionally, it could be stated that when 
coupled with a rigorous training program (like RPL School), the level of qualification 
previously held by the student is not as important as might have been assumed. This is a 
positive aspect of this interactive system in that its employment in the fleet will be to 


train Sailors of all ranks, rates and levels of experience. 


B. RECOMMENDATIONS FOR IDCTT (RPL) IMPROVEMENT 


IDCTT (RPL) is a very easy system to use, and the computer on which the 
software was installed was fast and capable. However, the limited number of scenarios 
may accelerate the student’s loss of interest in conducting the number of repetitions 
required to become a proficient Repair Party Leader. The following are 


recommendations for IDCTT (RPL) improvement. 


le Improve the feedback system. 

Currently, the post-scenario feedback provided to the student consists of a brief 
pass/fail assessment of performance, followed by a review of actions taken by the user. 
This could be embellished by designing the system to reinforce actions taken correctly, as 
well as pointing out procedural errors, steps taken out of sequence and errors made due to 
lack of knowledge regarding ship’s systems. A major portion of the debrief/critique 
presented at the conclusion of an actual damage control drill addresses what the Repair 
Locker did correctly, as well as presenting discrepancies. An enhancement of the 
feedback system would allow either supervisory personnel or the motivated user to tailor 


a training package to address shortcomings discovered during the scenario. For greatest 
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impact, IDCTT (RPL) could provide a printed report of errors, the action that should have 


been taken, and the reference for the step. 


Qi Simplify the process required to locally develop scenarios. 

Scenarios available today are provided by the prime software contractor. Making 
it possible for the shipboard damage control organization to locally generate scenarios 
would be a noteworthy enhancement in utility of the system aboard ship. The ability to 
build and test scenarios would provide at least two benefits. First, the ship would be able 
to provide tailored training to its Repair Locker Leaders. Second, the Damage Control 
Training Team would be able to prepare scenarios for live drills, and then test them for 
pitfalis prior to actually placing ship’s company at General Quarters and executing the 
drill plan. This would decrease time spent developing quality drills, and would enhance 
the amount of benefit damage control personnel would gain per hour spent at General 
Quarters. A well-designed drill makes a big difference in the training received by the 


Crew. 


3. Add a tutorial package to the existing software. 

Alessi and Trollip (1991) describe training as a four-part process. In its current 
configuration, IDCTT (RPL) does not address the aspect of “presenting the information” 
to the student. Addition of an in-depth tutorial covering advanced damage control theory 
and procedures is necessary in order to increase the training ability of the system 
Ideally, students would be able to select a training package covering a particular aspect of 
damage control, and, after taking the tutorial, immediately apply the material in the 
tutorial to a scenario selected from the menu. This rapid use of knowledge would serve 


to reinforce what has been learned, increasing the amount retained by the user. 


4. Add an on-line library. 
A noteworthy shortcoming of IDCTT (RPL) im its current configuration is the 
lack of an on-line library of damage control publications. While many actions required 


of the RPL should be committed to memory, many others allow for thought and review of 
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information. In a shipboard repair locker, the RPL has access to many sources of 
information, including a large library of pertinent damage control publications. By being 
able to access these during a scenario, the training environment would be more realistic, 
and would also familiarize the RPL with reference assets contamed in his or her repair 


locker. 


DB: Provide scenarios for other ship classes. 

Currently, IDCTT (RPL) scenarios exist only for DDG-51 class ships. While this 
class of ship comprises a significant portion of the Navy’s surface combatants, there are 
numerous other classes that are underrepresented. In order to enhance the fleet-wide 
utility of IDCTT (RPL), scenarios based aboard cruisers, guided missile frigates, 


amphibious and combat logistics force ships should be developed as soon as possible. 


Cc RECOMMENDATIONS FOR FURTHER RESEARCH 


This study provides evidence that using IDCTT (RPL) as an educational augment 
to the traditional Repair Party Leader School curriculum would be of benefit to the 
students enrolled in the course. There are, however, several additional areas of research 


that could be pursued in order to validate the findings of this study. 


d Improve the experimental design. 

In order to provide a better measure of effectiveness for IDCTT (RPL) as tested at 
the Repair Party Leader School, the pretest should take the form of an oral board 
comprised of questions similar to those asked at the end of the course. This change 
would provide a better indication of entrance level of knowledge pertinent to the type of 
learning that would be expected to be delivered by IDCTT (RPL). As conducted, it was 


difficult to relate pre-test scores and oral board results. 
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2. Other research opportunities. 

Aside from ensuring that the Repair Party Leader is fully competent at the 
completion of the qualification process, the most important factor is the amount of time 
required to achieve shipboard RPL certification. Other studies, including Army- 
sponsored research regarding the training of communications specialists (Winkler and 
Polich, 1991), report that interactive training systems significantly reduce the time 
required for a trainee to develop a skill. If students attending RPL School were 
identified, traed with IDCTT (RPL) as a supplement to traditional training, and tracked 
to completion of qualification aboard ship, the time required to complete this process 
could be compared with those not receiving IDCTT (RPL) training. This information 
could be used as support for a cost-benefit study on which to base procurement and 
system fielding decisions. Finally, an additional study could be commissioned to 
determine whether the effect was due to instruction or could be attributed directly to use 
ofthe IDCTT (RPL). While the result of this research is encouraging with respect to the 
prospect of using interactive methods to train Repair Party Leaders, it is still not clear 
what the difference will be when used in fleet applications. A study conducted aboard 
ship, comparing the actual performance of school graduate RPL’s with those trained 
using only IDCTT (RPL) would provide additional information relative to the system’s 


effectrveness. 


67 





APPENDIX A. IDCTT (RPL) TEACHING POINTS 


Dr. Eric Allely of Tekamah Company provided these teaching points. Tekamah 


develops the source code for all scenarios used in IDCTT (RPL). The listed teaching 
points were current as of March, 1997. 


Teaching Point Pitfall: action not taken 


1. 


Report Repair 3 Manned and Ready 


The Manned and Ready report signifies that personnel are available to combat 
casualties and are prepared with proper equipment, personnel protection and 
communications. It is imperative that the senior person in the Repair Station 
provides the report to DC Central as soon as personnel are ready to commence 
fighting the casualty. (NWP62-1D) 


Manned and Ready time requirements are contained in FXP-4 exercise MOB-D- 
3-SF. For a DD/DDG class ship the time standard for Manned and Ready is 5 
minutes. 


Zebra set. 


Material Condition zebra provides the greatest degree of subdivision and 
watertight integrity to the ship. It is the maximum state of readiness. 


In FXP-4 exercise MOB-D-11-SF, the time standard for setting material condition 
zebra for a DD/ DDG class ship is 5 minutes 


Modified Zebra 

Certain situation may require the CO to order material condition modified zebra 
set. Modified material condition zebra will give a higher survivability stance than 
condition yoke, while modified zebra is less restrictive and will allow the 
accomplishment of certain operational requirements.(DC PQS) 


Changing material condition (Y, W, circle W) 


Changes in the ship's material condition require Commanding Officer approval 
(NWP 3-20.31) 


Investigation 
Investigation teams should be rapidly dispatched. Any delays in investigating 
damage may result in the further progressive spread of damage. Investigation 


shall be thorough, conducted with caution, clearly and quickly reported and 
investigation is to be repeated (DC PQS) 
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10. 


Le 


12. 


13. 


Flooding Boundaries 


The setting of flooding boundaries should be an immediate action when flooding 
is reported to limit progressive flooding into adjacent compartments. 


Flooding boundaries should be set at the main transverse watertight bulkheads 
closest to the flooding. (DC PQS) 


Fire Main valve isolation 

Rapid firemain valve isolation will reduce flooding from ruptured piping. 
Firemain isolation should be coordinated with DC Central to prevent disruption to 
vital systems served by the firemain. 


Hole patched 


Failure to patch a hole in the bulkhead will result in progressive flooding into 
adjacent compartments. 


Progressive flooding 
Shore Bulkhead 


Failure to shore a panting bulkhead may result in the loss of the bulkhead and 
damage to the adjacent compartment. 


Indications of the need for shoring include deep bulges in plating, bowed frames 
and stanchions, loose rivets, cracked seams, and panting bulkheads. Panting is a 
dangerous condition. It results in metal fatigue. Which eventually causes cracking 
and splitting. (Repair Party Manual) 

When in doubt always shore! (Repair Party Manual) 

Request additional personnel for Flooding 

Requests for additional personnel should be coordinated via Damage Control 
Central. A failure to keep DC Central informed and to coordinate limited Damage 
Control resources may result in repair teams being overwhelmed by cascading 
casualties. 


Request route 


Routing of personnel transiting the Repair Station area should be coordinated with 
DC Central. 


Stability is Critical 
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14. 


i: 


16. 


The "Basic Thumbrules of Critical Stability" are contained in the CNSL/CNSP 
Repair Party Manual. All Repair Party Leaders should be familiar with basic 
stability concepts. In some circumstances, particularly when inport, an RPL may 
be the only person onboard with training in stability principles. 


Stability may be is critical if any of the following conditions exist following 
damage: 

Small or negative metacentric height 

exceeding floodable length 

excessive list 

heavy winds and seas 


Small or NegativeMetacentric Height 

When stability is critical due to small or negative metacentric height: 
a. The ship is logy with a slow erratic roll period and a tendency to hang at 
the end of the roll. 
b. The ship has a tendency to list at the same angle to either side. 
c. The ship has a list which cannot be accounted for by off-center weight. 
(Repair Party Manual) 


Exceeding Floodable Length 

When Stability is critical due to exceeding floodable length, the extent of flooding 
approaches or exceeds the maximum amount of flooding tabulated in the ships 
Damage Control Book, section II (a) (Repair Party Manual) 

Excessive List 

When Stability is critical due to excessive list, the ship lists to a static heel of 15 
degrees or more. (Repair Party Manual) 

Heavy Winds and Seas 

When Stability is critical due to heavy winds and seas, Heavy winds and rough 
seas are prevailing or are anticipated. (Repair Party Manual) 

Counterflooding or Ballasting 


Counterflooding or Ballasting requires the approval of the Commanding Officer 
(NWP 3-20.31) 


Jettisoning 
Jettisoning requires the approval of the Commanding Officer (NWP 3-20.31) 
Starting/Stopping Firepumps 


Firemain management requires close coordination with DC Central. The starting 
and stopping of fire pumps should be ordered by DC Central. 
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18. 


19. 


20. 


os 


22 


23. 


24. 


Zo. 


Dewater using installed Eductor (No FM) 

Installed eductors require proper firemain pressure to operate. Improper 
alignment of the ship's firemam system when using eductors may result in 
accidental flooding rather than dewatering of a compartment. 

Fire Boundaries 

Any physical barrier can be a fire boundary. Ideally fire boundaries are the 
bulkheads, deck, and overhead surrounding the fire. Secondary boundaries are 
generally set at watertight subdivisions or airtight boundaries. (NWP 62-1D) 


Smoke Boundaries 


Smoke boundaries are set to conta smoke within a fixed area and prevent the 
spread of smoke either horizontally or vertically. (NWP 62-1D) 


Failure to Request additional personnel for F/F 

Rapidly manning fire teams may be difficult if other Damage Control procedures 
are already in progress and repair personnel have been assigned to other teams. 
Emerging fire team requirements should be coordmated with DC Central to 
ensure the fastest response and prevent fires from rapidly spreading. 


Electrical Isolation 


The extent of compartment electrical isolation and the securing of lighting is 
typically determined by the scene leader.( NSTM 555) 


Mechanical Isolation 

When a space is abandoned due to fire, the space should be mechanically and 
electrically isolated to the greatest extent possible. A CO may choose not to 
isolate such spaces if they are essential to the safety, mobility, or fighting 
capability of the ship (NSTM 555). 

Securing of ventilation 

The decision to secure ventilation systems is made on scene.(NWP 62-1D) 


Magazine Sprinkler activation 


Activation of magazine sprinklers requires the approval of the Commanding 
Officer. (NWP 3-20.31) 


Magazine sprinklers activation, no FM 


a2 


26. 


2. 


28. 


29. 


30. 


31. 


Sip 


Firemain management requires attention to vital systems served by the firemain 
including magazine sprinkler systems. 


Movement of Ammunition 

The movement of ammunition onboard the ship requires approval from the 
Commanding Officer. Ammunition movement should be conducted by trained 
Combat Systems personnel and should be coordinated via DC Central and 
Combat Systems Maintenance Central. 

Halon activation 

Halon activation may prove ineffective if the compartment has been sufficiently 
damaged to prevent Halon from reaching the proper concentration in the space to 
extinguish the fire. 

Jettison of ammunition 

The jettisoning of objects over the side requires Commanding Officer approval. 
(NWP 3-20.31) The movement of ammunition (including moving it over the 
side) also requires Commanding Officer approval. 


Jettison of Gas Cylinders 


The jettisoning of objects over the side requires CO approval. (NWP 3-20.31) 
Evacuation of Repair 3 (smoke) 


The evacuation of personnel within the Repair Station area to the weatherdecks 
may be required when smoke boundaries are not correctly ordered and set. 


Direct Attack 


In a direct attack, firefighters advance into the immediate fire area and apply the 
extinguishing agent directly onto the seat of the fire. (NSTM 555) 


Indirect Attack 

An indirect attack is the application of water fog into the fire space through an 
existing access or through a hole cut in a bulkhead or overhead when heat or other 
conditions deny access to the space, an indirect attack may improve condition 


conditions to permit reentry for a direct attack.. (NSTM 555) 


Active Desmoking 
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33. 


34. 


55: 


36. 


3. 


38. 


39. 


Active desmoking is removing smoke and heat from the smoke control zone prior 
to extinguishing the fire, to aid firefighting efforts and reduce smoke spread in the 
ship. (NSTM 555) 


Overhaul of Fire 


Overhaul of a fire is an examination and cleanup operation. It includes finding 
and extinguishing hidden fires and determining whether the fire has extended to 
other areas of the ship. (NSTM 555) 


Desmoke 


After a compartment fire has been extinguished, combustible gases may be 
present. The goal of desmoking is to replace 95% of the smoke laden air with 
fresh air. This will require approximately 4 complete space volume changes in the 
compartment. (NSTM 555) 


Atmospheric Testing 

All spaces should be desmoked before atmospheric testing is started because 
oxygen sensors do not operate reliably if the sensor is exposed to excessive 
moisture or is in contact with postfire atmospheric particulates. 

Request Personnel Casualty Transport Route 

FXP-4 exercise FSO-M-2-SF states that a safe route for stretcher-bearers should 
be received from DC Central. The FXP-4 time standard for stretcher- bearers to 
arrive on scene is within 10 minutes of the casualty report. 

Manning Flight Quarters 

Due to damage control actions already in progress in the Repair 3 area, manning 
the Aviation Crash and Salvage Team exclusively from Repair 3 may result in 
Repair team manning shortfalls. As RPL you should inform DC Central of 
anticipated problems and request additional personnel when necessary. 

Hazardous Material Incident Response 

Any material that because of its quantity, concentration, or physical or chemical 
characteristics may pose a substantial hazard to human health or the environment 
when released or spilled. (DC PQS) 


Casualty Power 


The engineering officer will designate the switchboard and riser to be used as a 
source of casualty power. (Repair Party Manual) 
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40. 


41. 


42. 


43. 


The casualty power route should be requested from DC Central. 
Permission to rig casualty power should be requested from DC Central. 
MOPP Levels 


Changes to the MOPP level should be directed by DC Central and the 
Commanding Officer. 


Counter Measure Washdown System 


Activation of the counter measure washdown system (CMWDS) requires the 
approval of the ships Commanding Officer. (NWP 62-1D) 


Dispatching External CBR Monitors 


Dispatching external CBR monitors outside the skin of the ship when in a CBR 
environment requires the Commanding Officer's approval (Repair Party Manual) 


Dispatching External CBR Decontamination Teams 
Dispatching External CBR Decontamination Teams outside the skin of the ship 


when in a CBR environment requires the Commanding Officer's approval (Repair 
Party Manual) 
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APPENDIX B STUDENT ENTRANCE DATA SHEET AND PRE-TEST 


This appendix provides the pre-test administered to students in the March, Apnil 
and May 1997 Repair Party Leader School at Fleet Trammg Center, San Diego California. 
The first page of this examination provided the vehicle for collection of demographic data 


on each participating student. 
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REPAIR PARTY LEADER COURSE PRE-TEST 
INSTRUCTIONS FOR TAKING TEST 
READ CAREFULLY 


1. Purpose of the Test. This pre-test is given in support of research that is being done 
regarding the possible incorporation of interactive courseware in the training process at 
this school. Your participation in this study is purely voluntary, but you may have fun and 
really learn something! This test is given in order to determine the baseline level of 
knowledge each student has upon enrolling in the course. Your score on this exam will 
have no impact on the remainder of the course. The questions for this test will be multiple 
choice, true-false, and fill-in. Circle the correct answer or fill in the blank as required. 


2. Quickly fill m the mformation requested on the attached mformation sheet. 


3. If you need assistance during the test, raise your hand. One of the mstructors will 
assist you. 


4. Examples 
A. MULTIPLE CHOICE AND FILL-IN 


Each question has four choices; a, b, c, and d. Read all four choices carefully. Only 
one answer will be correct. 


B. TRUE-FALSE 


If any part of the statement is false, the whole item is to be marked false on the test. 
True statements are to be marked by circling ‘True’ on the test sheet. For false 
statements, circle ‘False’. 


5. 20 minutes is allotted for this test. When you have completed the test, remam quiet at 
your seat until told to turn in your test. 


6. CAUTION 


If you desire to change an answer, erase your first choice completely, then select your 
new choice. ANY CHOICE HAVING TWO OR MORE ANSWERS SELECTED IS 
AUTOMATICALLY CONSIDERED WRONG. ANSWER EVERY QUESTION. IF 
YOU UNDERSTAND THE INSTRUCTIONS, WAIT FOR THE SIGNAL, AND THEN 
BEGIN THE TEST. 
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STUDENT INFORMATION SHEET 


REPAIR PARTY LEADER COURSE 


Date 1997 
Name Rank/Rate ESWS Y N 
(circle one) 
SSN Command 
(name and hull number) 

Date Reported to Present Command (Month/Year) 
How much total sea time have you accumulated? years months 
Where do you go now for GQ? 
What is your job there? 
Have you ever attended 

General Shipboard Firefighting School Y N 

Advanced Shipboard Firefighting School Y N 

DCPO School Y N 

Buttercup Wet Trainer Y N 


1. Fires on ships underway that have no at-sea fire party will normally be handled by the 


a. duty section. 

b. m-port fire party. 

C. repair party personnel. 

d. Naval Station fire protection personnel. 1.1.1 


2. The Repair Party Leader's duties and responsibilities include 


establishing maximum permissible exposure (MPE). 

ordering the activation/deactivation of the water wash down system (WWDS). 
ordering the flooding of magazines. 

assigning repair party personnel to teams. NeeZ 


ao 


3. Part II of the Ship's Damage Control book contains 


a. general mformation. 
b. damage control systems. 


19 


c. miscellaneous systems. 
d. electrical systems. een 


4. Heavy lines on isometric damage control diagrams indicate 
boundaries. 


water-tight and oil-tight 

air-tight 

fire retardant and flame-tight 

non-tight 12a 


no op 


5. A DC training simulation of black cloth taped over an item would indicate 


a. black smoke. 

b. spilled oil. 

c. charming. 

d. class "B” fire. 1.3.1.1 


For items 6 through 9, match the advantages/lumitations in Column A to the DC 
communications systems in Column B by drawing a line between the two. 


COLUMN A COLUMN B 


6. A flexible and survivable means of a. Intercommunication system. 
rapid communications but can not be 
used m a weapons handling imcident/ 
accident. b. DC WIFCOM. 


7. Requires no external source of power. c. Ship's service telephones (J-circuit) 
8. Provides a fast and dependable two 
way transmission between DCC and each _ d. Sound-powered telephone 


system repair station. 


9. Not rugged, may go out of commission 
early mn action, (requires electrical power). 


10. Select the priority use of communications from DCC to the repair lockers. 


a. 3,1,2 and 4. 1. DC Wifcom 

b. 1,3, 2 and 4.. 2. MC Circuit. 

c. 3,2,4 and 1 3. Sound Powered Telephones. 

d. 1,3,4and2 4. Ship's Service Telephone, (J-circuit). 
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11. Isometric DC Diagrams, Status Boards, and Logs 


. are all unclassified documents. 

require constant updating and close attention. 
seldom require updating. 

are only located on the bridge and in DCC. 


ao of 


12. A proper wedge has alengthof _ times its butt thickness. 


a. © oe & 
mt CGCO CN W 


0 
13. Select the first step for by-passing a rupture in the firemain. 


a. Rig one 2 1/2 inch hose from the two nearest pressurized sections beyond the 
rupture. 

b. Remove the wye-gate from the required fireplugs. 

c. Isolate the ruptured section. 

d. Shut down the fireplug nearest the rupture. 


14. The maximum length of a shore can not be more than times it's minimum 
thickness. 


15. An electric submersible pump will discharge _ | GPM with a 70 foot static head. 


140 
160 
200 
. 250 


ae oP 


16. The permanent angle of heel taken by a ship due to flooding is called 


a. roll 
b. trim 
c. pitch 
d. list 


$1 


17. 


18. 


i: 


20. 


PAs 


22. 


The secondary dramage system 


a. drains spaces forward and aft of the main machinery compartments. 

b. has larger piping than the main drainage system. 

c. is connected to and uses the same pumps and eductors as the main drainage 
system. 

d. drains the main machmery compartments. 


The firemain provides operating pressure to eductors in the systems. 


main and secondary drainage 
. supply and exhaust 

primary and alternate 

. auxiliary firemaim 


a omens 


For desmoking after a fire, installed or fixed ventilation systems can 


a. never be used. 

b. be used with the repair party leaders permission. 

c. be used with restrictions. 

d. be used ifthe exhaust discharges into a different space. 


The Ram Fan 2000 is driven by 


. LP air 

. Water under pressure 
110 VAC 

. HP air 


$5 


=O lS 


Extra casualty power cables are stored m DC repair lockers in lengths of feet. 


100 
150 
. 200 
250 


Oo om & 


. 


The function of the Casualty Power System is to 


a. provide power to all weapon systems. 

b. supply power to all of the ships electrical equipment. 

c. make temporary power connections to vital systems/equipment. 
d. splice and repair permanently damaged electrical components. 
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23. What is the correct sequence to be followed when rigging casualty power cable? 


a. Rig power source to load 

b. Rig from load to power source 

c. No preferred sequence 

d. From switchboard to power panel 


24. An itemized list of all damage control equipment located in a compartment is called 
a/an 


a. Damage Control Book. 

b. Closure Log. 

c. Compartment Check-Off List, (CCOL). 
d. EOSS. 


MATCH THE COMPLETED DAMAGE CONTROL MESSAGE TO THE 
CORRESPONDING LETTER CODE THAT CORRECTLY IDENTIFIES THE 
MESSAGE. 


25. Message No. 1 


a. Un-jammed WTH 2-35-2 in compartment 2-23-0-L. 

b. Un-jammed quick acting scuttle 2-35-2 in compartment 2-23-0-L. 
c. Jammed WTH 2-35-2 in compartment 2-23-0-L. 

d. Un-jammed quick acting WTH 2-35-2 in compartment 2-23-0-L. 


26. Message No. 2 


a. Electrical power lost in compartment 3-23-0-E. 

b. Secured communications to compartment 3-23-0-E. 

c. Electrical power ordered secured to compartment 3-23-0-E. 
d. Restored communications to compartment 3-23-0-E. 


27. Message No. 3 


a. Minus oxygen, plus toxic gases in compartment 3-23-0-E. 

b. Plus oxygen, plus explosive gases and plus toxic gases in compartment 3-23-0-E. 

c. Plus oxygen, minus explosive gases and minus toxic gases in compartment 
3-23-0-E. | 

d. Plus oxygen, minus toxic gases in compartment 3-23-0-E. 


28. Message No. 4 


a. Watertight hatch 3-36-2 is shored and shoring watch is set in compartment 3-23- 
0-E. 
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b. Watertight door 3-36-2 is reported shored in compartment 3-23-0-E. 

c. Watertight hatch 3-36-2 is sprung and shoring watch ts set in compartment 3-23- 
0-E. 

d. Watertight hatch 3-36-2 in compartment 3-23-0-E is reported damaged. 


29. Message No. 5 


. Jammed watertight hatch 2-36-2 in compartment 2-24-0-L. 

. Un-jammed WTH 2-35-2 in compartment 2-23-0-L. 

. Jammed watertight hatch 2-35-2 in compartment 2-23-0-L. 
Un-jammed watertight door 2-36-2 in compartment 2-24-0-L. 


Oo Ss & 


= 


30. Message No. 6 


a. Ruptured firemain riser reported in compartment 2-15-1-L. 
b. Ruptured firemam riser is repaired in compartment 2-15-1-L. 
c. Desmoking completed in compartment 2-15-1-L. 

d. Restored flooding boundaries in compartment 2-15-1-L. 
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Message Nr 1 


Scene L dr 
Bridge 


Investigator 
3-381-1-Q 


OVHD/FWD 


Dec k/Aft 





NWP 3 - 20 . 31, Fi gure 4. 3, Sheet 9 
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Message Nr 2 


Investigator 
3-90-2-L 
FR 96 


4 inch, FR 96, Stbd, 2 f to f f deck 


OVHD/FWD 


Dec k/Aft 





NWP 3-20 . 31, Fi gure 4. 3, Sheet 13 
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Message Nr 3 


OVHD/FWD 


Dec k/Aft 





NWP 3-20. 31, Figure 4. 3, Sheet 9 


$7 


Message Nr 4 


sceneLdr 
Bridge 


Investigator 


OVHD/FWD 


Dec k/Aft 





NWP 3-20.31, Fi gure 4. 3, Sheet 1 
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APPENDIX C ORAL BOARD QUESTIONS AND PROCTOR INSTRUCTIONS 


This appendix provides the instructions given to proctors administering the end-of- 
course oral boards, as well as questions to be asked verbatim. For most of the oral board 
questions, the applicable line item from NAVEDTRA 43119-G, Watchstation 318 (Repair 
Party Leader) has been identified. The question is designed to determine the individual’s 
knowledge regarding that particular line item of the PQS. 
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Instructions to Oral Board Proctor 


The purpose of this oral board is to provide a post-test to students of the Repair Party 
Course that may assist in determining the value of the IDCTT (RPL) as a system with 
which to augment damage control training delivered in the schoolhouse. The following 
instructions must be followed explicitly in order to ensure quality and validity of data 


collected. 


Read the following paragraph to the student. 


“The purpose of this oral board is to gather data that will help m determming the value of 
the interactive courseware that has been installed here at FTC. This board will have no 
impact on your final grade here at the school. I will now go over the method I will use for 
conducting this board. 


1. You are to relax...remember, this board doesn’t count toward your score in 
the course. 

2. Do your best to answer each question. Think about your answer before giving 
it. 

3. This board will last about 15 minutes. I will ask you a question every minute, 
on the minute. If you are still answering the previous question, I will stop you 
and we will move to the next question. 

4. During the board, I will answer no questions and will not provide any 


information other than the question. 


Before we begin, do you have any questions? Ifnot, we'll start the board.” 


Reminders for the Proctor 


1. Do not ask the student if he/she was in the group that used IDCTT. 
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Do not let the student see you writing your observations (i.e. don’t jump on a 
response and make a production out of recording it) 

Ask the questions at exactly 1 minute intervals. If the student answers the 
previous question early, wait until 1 mimute has elapsed, then ask the next 
question. Ifthe student is in the middle of an answer at the end of the 1 minute 
period, stop him/her and ask the next question. 

Record the student’s correct responses by placing checks in the blocks 
corresponding to the response. You have the authority to interpret the answer 
given in order to decide if it is correct (i.e. it is understood that the student will 
probably not quote the reference verbatim). 

Do not enhance/embellish the question. These questions are scripted and must 
be read verbatim to each student. 

Use a new sheet for each student. Ensure you record the student’s SSN on the 
sheet prior to starting the board. 

It is most critical that the boards be consistent from one student to the 


next. 
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Repair Party Leader 
Oral Board Questions 





Student Name Board Date 

SSN 

NAVEDTRA 43119-G Topic Task 

318.2.19 Maintain communications with DC Say steps of procedure 


Central and other Repair Stations. 
Question. You are Repair II Locker Leader. By ship’s instruction, your locker is next in 
succession for control if something happens to DC Central. How will you ensure good 


communications and control of the ship-wide DC situation? 


Answer. (Reference NSTM 079V2, 079.38.173) 





1. Conduct periodic communications checks with DCC. 
2 Maintain a plot of damage in all areas of the ship. 
a Take positive control if DC Central is out of action. 
NAVEDTRA 43119-G Topic Task 
318.2.12 Coordinate Repair Party efforts to Say steps of procedure 
combat/control flooding. 


Question. List the actions the Repair Party Leader would take if flooding were reported 
in his/her area of responsibility. 


Answer. (Reference CNSLINST 3541.1C/CNSPINST 3541.4B) 
1. Establish flooding boundaries. 
Electrically isolate the affected space. 
Mechanically isolate the affected space. 
Prioritize dewatering efforts. 
Evaluate the stability situation. 


RR 


eee aa 





Question. What does this message tell you as Repair Party Leader? (Show Message 
Blank Nr 1) 


Answer. (Reference NWP 3-20.31, Figure 4.3, sheet 9) 
Message Nr | says electrical power isolated im compartment 3-381-1-Q 


C2 





Question. What are the minimum responsibilities of the Repair Party Leader in the event 
of fire in his/her area of responsibility. 


Answer. (Reference NWP 3-20.31, Chapter 9.2.1.1) 


1. Ensure boundaries are established. 

Dt Coordinate efforts to prevent vertical fire spread. 

3. Provide logistic support to the scene (personnel and 
equipment) 

4. Maintain an accurate plot. 

B) Make required status reports. 





Question. List all the damage control actions for which you would need to get 
permission prior to doing. 


Answer. (Reference Repair Party Manual, Chapter 1, Art 102, NWP 3-20.31, 





Art 1.5.8) 
le Ballasting/deballasting 
2: Counter flooding 
8 Changes to established material condition of readiness 
4. Flooding weapons magazines 
a Desmoking using installed ventilation 





Question. What does this message tell you as Repair Party Leader? (Show Message 
Blank Nr 2) 


Answer. (Reference NWP 3-20.31, Figure 4.3, sheet 13) 
Message Nr 2 says a 4 inch hole, 2 feet off the deck in compartment 3- 
90-2-L at FR 96, starboard side is being plugged/patched. 





NAVEDTRA 43119-G Topic Task 
318.2.9 Coordinate Repair Party effortsto Say steps of procedure 
combat/control fire. 


Question. List the steps the Repair Party Leader would take in fighting a fire in his/her 
area of responsibility. 


Answer. (Reference NSTM 555-5.3.2) 
1. Determine the location of the fire. 
Determine the class of the fire. 
Determine the hazards the fire poses to adjacent spaces. 
Order fire boundaries. 





= 
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Order electrical isolation. 
Order mechanical isolation. 
Order smoke boundaries. 
Attack the fire. 


7 
6. 
te 
8. 





NAVEDTRA 43119-G Topic Task 
318.2.13 Coordinate Repair Party effortsto Say reasons for step 
combat/control structural damage. 


Question. List the reports or indications the Repair Party Leader might get that would 
cause him to believe shoring is required. 


Answer. (Reference NSTM 079V2, 079-44.125) 
i. Deep bulges in plating. 

Bowed/bent frames. 

Bowed/bent stanchions. 

Cracked seams. 

Panting bulkhead. 


LL 


i ee 





Question. The Scene Leader is responsible for deciding whether to have the firefighters 
wear the FireFighter’s Ensemble. You have a new scene leader who asks you whether his 
teams should wear the FFE. What factors should you consider in your response? 


Answer. (Reference NSTM 555-5.2.1) 


1. Reduced endurance due to heat stress. 
2. Size of the fire (i.e. are the FFE’s necessary for a small fire?). 
3. Progress of the fire while the fireparty is dressing out 


(i.e. is the fire likely to spread if the fireparty is delayed?) 





NAVEDTRA 43119-G Topic Task 
318.2.13 Coordinate Repair Party effortsto | Say reasons for step 
combat/control structural damage. 


Question. Why is a panting bulkhead a dangerous condition? 
Answer. (Reference NSTM 079V2, 079-44.125) 


Panting is a dangerous condition as it causes metal fatigue which in time 
will result in cracking or splitting. 





Question. What does this message tell you as Repair Party Leader? (Show Message 
Blank Nr 3) 
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Answer. (Reference NWP 3-20.31, Figure 4.3, sheet 7) 
Message Nr 3 says Progressive flooding reported in compartment 2-406- 


1-Q, 4” deep 
NAVEDTRA 43119-G Topic Task 
318.2.9 Coordinate Repair Party effortsto Say steps of procedure 


combat/control fire. 


Question. What are your methods of communication from the Repair Locker to other 
Stations? 


Answer. (Reference NWP3-20.31, Section 4.1) 


ile Sound-powered phones 

2 Ship’s Service Telephones 
3. Intercom (i.e. 4MC) 

4. DC WIFCOM 

>. Messengers 


Question. Your scene leader calls back to the Locker and reports that he needs 3 OBA 
relief personnel. You look around the Repair Locker area and there are no extra 
personnel available. What action do you take to help out the scene leader? 


Answer. Call DC Central and ask the DCA to find and send additional 
personnel. 





Question. What does this message tell you as Repair Party Leader? (Show Message 
Blank Nr 4) 


Answer. (Reference NWP 3-20.31, Figure 4.3, sheet 1) 
Message Nr 4 says fire boundaries have been set, primary forward FR 
200, primary aft FR 245, secondary forward FR175, secondary aft FR 270. 








Question. When might stability of the ship be considered a problem? 


Answer. (Reference NWP 3-20.31, Art 5.3.5) 
Ls Small or negative metacentric height with a logy, long period roll 
The extent of flooding exceeds floodable length 
Excessive list 
Heavy winds and seas combined with existing flooding 





ep 
a 
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APPENDIX D. DATA SETS 


This appendix contains the data sets used for this study. To simplify the analysis, 
integer indicators were assigned to some of the variables. Table 9 lists the variable and 


the applicable integer indicator employed. In order to incorporate education and rank 


Variable Integer Indicator _ 
Class Convening Date (Mar 17, 1997) 1 
Class Convening Date (April 14, 1997) Zz 
Class Convening Date (May 12, 1997) 3 
Control Group 0 
Treatment Group ] 
Male ] 
Female 0 
Enlisted Surface Warfare (ESWS) Qualified ] 
Not ESWS Qualified 0 


Table 9. Assignment of Integer Indicators. 


into the analysis, integers indicators had to be developed for those two variables. In the 
case of education, the DMDC database already had values associated with particular 
levels of education (Table 10). Table 11 provides the indicators assigned to the rank/rate 


variable. 


Education Variable Integer Indicator 
3-4 Years High School (No diploma) 5 
High School Diploma 6 
2 Years of College 8 
College Graduate 10 
Masters or Equivalent 11 


High School GED | 13 


Table 10. Education Variable Integer Indicators. 
In the data set, E-5 represents second class petty officers. These subjects can be 
identified as being those whose rate designator ends with “2” (e.g. AT2). First class petty 


officers are those whose rate designator ends with “1” (e.g. HT1). E-7 represents a Chief 
Petty Officer (e.g. SKC), while W-2 corresponds with warrant officers (e.g. CWQ2). 
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Among commissioned officers, O-1 and O-2 represent Ensign (ENS) and Lieutenant 


(junior grade) (LTJG), while O-3 and above corresponds with Lieutenants and senior. 


Rank/Rate Variable _ Integer Indicator 


E-5 
E-6 


E-7 through W-4 
O-1 through O-2 


O-3 and above 


Table 11. Rank/Rate Variable Integer Indicators 


] 


2 
3 
4 
5 


Table 12 provides the complete data covering all three classes that participated in 


the study between March and May 1997. The variable “Sea Time” refers to the amount 


Class 
Date 


3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
3/17/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 


Subject 
Nr 


On A MA Bh WN — 


NW NWN NWN WN DNDN KN = = BS BM RB eB ES eS 
IUA~OAUBRWNHHK OM WMIUDAWNWRWNHKO MY 


| Group 


Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Treatment 
Treatment 
Treatment 
Treatment 


Rank/ Education ESWS Gender SeaTime PreTest ExpTime OBS] 


Rate 


(mo) 


(min) 


AT2 
AO] 
DC2 
SMC 
ENC 
ENS 
LT 
ENS 
ENS 
ENS 
CWO3 
LTJG 
DM] 
LCDR 
RMC 
CWO3 
BM2 
SKC 
MMC 
HT1 
LTJG 
ENS 
LTJG 
GSM2 
EM1 
HT1 
GSE2 


NAA A 


= Sr anwaoa 


NNN AW 


ZA2ZKSAZAASASmYK KKK Y™S™Y2Z 4244424424424 44K KR K% 424 


Saas eS 2 Ses Se Se a 


19 
64 
31 
120 
94 
7 
43 
12 
8 
0 
2 
37 
50 
] 
75 
173 
120 
51 
111 
63 


1, 
52 
97 
1D 
6] 


66.67 
70.00 
76.67 
80.00 
46.67 
73.33 
733 
70.00 
(3.30 
70.00 
50.00 
80.00 
23.33 
60.00 
46.67 
76.67 
63.33 
46.67 
66.67 
80.00 
7333 
63.33 
70.00 
90.00 
7333 
70.00 
60.00 


9] 
6 
6 
6 
7 

36 
iS 

ae 

32 

34 

179 

17 


© 


See oO Oe. So oS > 


38.78 
44.90 
44.90 
67.35 
59.18 
53.06 
65.31 
57.14 
40.82 
59.18 
55.10 
28.57 
22.45 
46.94 
61.22 
61.22 
22.45 
40.82 
59.18 
63.27 
38.78 
43.98 
44.90 
57.14 
43.98 
53.06 
51.02 





98 


Class Subject Group Rank/ Education ESWS Gender SeaTime PreTest ExpTime OBS] 


Date 


4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
4/14/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
3/12/97 
3/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 
5/12/97 


Nr 


28 
29 
30 
31 
32 
33 
34 
35 
36 
a7 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 


Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Treatment 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 
Control 


Rate 


DCl 
ENC 
CTRC 
RMC 
ENS 
LTJG 
ENS 
ICl 
DCl 
HT1 
CWO3 
ENS 
MMC 
MM1 
HT2 
HTC 
PNC 
PR1 
EMC 
ICl 


DC2 
DCl 
PNC 
ENS 
LTJG 


GSM1 
SH1 
LTJG 
ENS 


LTJG 
ENS 


Table 12. Complete Data Set. 
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Soa ooo = oo 7 8 Ses Sesees = Sees SHS HSS SS SS eS Se Se 


(mo) 


92 
96 
DD 
0 
0 
1] 
Zz 
72 
64 
48 
162 
21 
96 
31 
29 
108 
36 
47 
66 
105 
0 
29 
38 
60 
0 
ZS 
Z 
102 
77 
24 
0 
31 
18 
44 
11 
6 
97 
25 
101 
25 
25 
34 
36 
96 
26 
10 


73.33 
70.00 
56.67 
70.00 
56.67 
90.00 
66.67 
60.00 
76.67 
46.67 
80.00 
73.33 
73.33 
76.67 
80.00 
76.67 
50.00 
70.00 
66.67 
76.67 
60.00 
70.00 
76.67 
70.00 
60.00 
76.67 
66.67 
70.00 
56.67 
76.67 
66.67 
60.00 
43.33 
60.00 
66.67 
86.67 
73.33 
70.00 
63.33 
80.00 
138 
63.33 
60.00 
40.00 
i236 
70.00 


(min) 


108 
6 
46 


eooocooo oo coc © © 


NN = 
No kh NO 


48 


No Pe eK wv 
NNO hN WN 


NON — — 
Seen OS Se 


44.90 
22.45 
51.02 
57.14 
44.90 
51.02 
46.94 
65.31 
51.02 
38.78 
61.22 
63.27 
foe 
36.73 
63.27 
53.06 
51-02 
40,82 
57.14 
53.06 
79.59 
93.88 
93.88 
ii2s 
$1.63 
69.39 
71.43 
93.88 
Joa 
75.51 
69.39 
77.55 
$1.63 
38.78 
Sz 
65.31 
34.69 
48.98 
40.82 
55.10 
36.73 
38.78 
42.86 
32.65 
44.90 
55.10 


of sea time the subject had accumulated (in months) up to the date the Repair Party 
Leader School class commenced. The PreTest column provides the score earned by the 
subject on the pretest administered the first day of the course. Exposure time is the 
amount of time (in minutes) the subject spent actually working scenarios on the IDCTT 
(RPL) computer. This was calculated by taking the difference between scenario start 
time and finish time for each scenario run by the members of the treatment group. Start 
and finish times are automatically recorded by the system and are stored on the 
computer's hard drive. The column labeled “OBS1” provides the score the subject earned 
on the post-treatment oral board conducted during the second week of the course. 


Table 13 provides data used for analysis of the May 97 class alone. 





Table 13. May 1997 Data Set. 
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Class Subject Group  Rank/ Education ESWS Gender SeaTime PreTest ExpTime OBSI 
_ Date Nr Rate (mo) (min) 

5/12/97 48 Treatment ENS 10 ) } ) 60.00 12 79.59 
5/12/97 49 Treatment DC2 6 ] 1 29 70.00 24 93.88 
5/12/97 SO Treatment DCl 13 1 1 38 76.67 ze 93.88 
5/12/97 51 Treatment PNC 6 1 1 60 70.00 48 Tiss 
5/12/97 52 Treatment ENS 10 0 ] 0 60.00 26 81.63 
5/12/97 53 Treatment LTJG 10 0 ] Zz 76.67 2 69.39 
5/12/97 54 Treatment ENS 10 0 1 2 66.67 14 71.43 
5/12/97 55 Treatment GSM1 6 1 1 102 70.00 48 93.88 
5/12/97 56 Treatment SHI 6 1 ] TA 56.67 12 7550 
5/12/97 57 Treatment LTJG 10 0 1 24 76.67 26 1538 
5/12/97 58 Treatment ENS 10 0 0 0 66.67 12 69.39 
5/12/97 59 Treatment LT 10 ) 0 31 60.00 10 Tis 
5/12/97 60 # £'Treatment LT 10 0 1 18 43.33 20 81.63 
S/12/97T—s 61 Control EN2 6 0 ] 44 60.00 0 38.78 
S/12/97 —s @2 Control LTJG 10 0 1 11 66.67 0 51.02 
5/12/97 ~—s 63 Control ENS 10 0 1 6 86.67 0 65.31 
5/12/97 = 64 Control DCl 6 ] 1 97 73.33 0 34.69 
5/12/97 —s 6 Control LTJG 10 ) 1 25 70.00 0 48.98 
5/12/97 —s- 66 Control DCl 13 1 1 101 63.33 0 40.82 
5/12/97 —s 67 Control LTJG 10 0 1 25 80.00 0 55g 
5/12/97 —s- 68 Control LTJG 10 0 1 25 73.33 0 36.73 
5/12/97 69 Control SKC 6 0 1 34 63.33 0 38.78 
5/12/97 70 Control HT1 6 0 0 36 60.00 0 42.86 
5/12/97 71 Control BMC 6 1 1 96 40.00 0 32.65 
5/12/97 —s «72 Control LTJG 10 0 1 26 73.33 0 44.90 
5/12/97 —s 73 Control ENS 10 0 1 10 70.00 0 55.10 


Table 14 provides the censored data set. As mentioned in Chapter IV.B.1, additional 
information was available for 52 of the 73 students participating in the study. The 
censored data set provided as Table 14 indicates the information available for these 
subjects. The data is listed by subject number; all information listed in Table 12 is the 
same. In the race category, | signifies white, 2 is black and 3 is other. The AFQT column 


Subject Nr Race AFQT Mental SAR SPC SCS CAI 


Cat 

1 1 93 1 64 53 53 56.67 
Z Z 42 3.5 47 61 51 53.00 
3 3 51 3 47 44 52 47.67 
4 ] 62 3 598 52 45 51.67 
5 Z ao 3 61 56 38 51.67 
6 1 98 ] 64 61 65 = 63.33 
7 1 69 2 Deen 7.00 
10 3 84 Zz 62 47 #£4x°71 ~~ 60.00 
11 ] 58 3 61 57 38 52.00 
12 3 66 2 58 59 46 54.33 
13 ji 33 aD 57 =638)— 43—S 46.00 
14 ] 78 2 62.55" "584 58:33 
16 ] 67 2 54. 37S 43— 44.67 
18 2 51 3 52°51 70 57.67 
19 ] 78 2 61 56 57 58.00 
20 ] 7 3 56 51 41 49.33 
21 3 94 ] 59 62 53 # 58.00 
24 ] 81 2 62 59 46 55.67 
ZS 1 TS 2 62 59 44 55.00 
26 2 50 3 52. 51 £68 57.00 
ZT 1 44 3168) 51 53 49 51.00 
28 ] ao 3 53. 51 45 49.67 
Zo 3 64 5 61 53 60 58.00 
30 1 fe) Z 62 60 50 = 57.33 
31 ] 33 3.5 45 50 44 46.33 
35 iz 47 35 57 49 61 55.67 
36 1 56 3 55 56 42 51.00 
37 3 17 4.33 50 42 42 44.67 
39 ] 83 i 63 61 46 56.67 
40 ] 87 2 64 55 £61 =~ 60.00 
41 ] 94 ] 65 54 69 62.67 
42 ] 78 2 59 56 58 57.67 
43 ] 54 3 56 55 42 51.00 
44 ] 45 3.5 48 50 49 49.00 
45 63 3 57 54 46 52.33 
46 ] 98 1 51 63 64 59.33 
47 1 86 2 66 64 45 58.33 
49 ] 84 2 61 62 68 = 63.67 
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50 1 71 a 58 59 57 58.00 
51 1 54 3 56 55 42 = 51.00 
55 ] 64 3 54. 58 53 55.00 
56 3 +4 325 56 50 54 53.33 
57 ] 94 ] 63 61 70 64.67 
6] 3 36 3.5 47 42 46 45.00 
63 ] 92 2 62 54 56 57.33 
64 ] 74 2 54. 54 65 = 57.67 
65 ] 72 a 59° Slaw 65 58.33 
66 2 35 3:3 51 50 45 48.67 
67 ] 96 1 64 56 (‘53 57.67 
69 yu 67 2 54. 56 48 52.67 
70 ] 76 Z 64 56 48 56.00 
7] 1 41 3.5 69 45 54 56.00 


Table 14. Censored Data Set. 


provides the percentile score achieved by the subject on the Armed Forces Qualification 
Test taken at the time of entry into the service. The Mental Category assigned to the 
individual is based on the AFQT percentile, as indicated in Table 15. SAR, SPC, SCS 
and CAI are as described in Chapter IV.B.2. The “S” indicates that the scores used were 


standardized by DMDC to account for variations between examinations. 


1 to 9 V > 
10 to 15 IV(c) 4.66 
16 to 20 IV(b) 4.33 
21 to 30 IV(a) 4 
31 to 49 Ifl(b) 35 
50 to 64 Ifi(a) 3 
65 to 92 II 2 
93 to 99 I ] 


Table 15. AFQT Scores and Associated Mental Categories. 
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APPENDIX E. STEPWISE REGRESSION RESULTS 


Stepwise regression was the primary tool used to aid in the selection of variables 
for this model. Further information regarding this technique is available in various 
books, including the reference listed in Chapter V.A. S-Plus was used to conduct the 
stepwise regression, and the results of this process is provided in this appendix. Table 16 
provides the results for the complete data set, while Tables 17 and 18 refer to the May 
1997 and Censored data sets, respectively. 
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Table 16. Stepwise Regression Results (Complete data set). 
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Class Group Rank Education ESWS Gender SeaTime PreTest ExpTime 


1(+2) F T F F F F F re F 


94.20182 4.905348 2.057581 1.780417 4.010695 0.677225 0.188848 0.000935 





Table 17. Stepwise Regression Results (May 1997 data set). 
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APPENDIX F. RESULTS OF ANOVA 


Analysis of variance (ANOVA) was used to refine the model suggested by 
stepwise regression. Table 19 is the output provided by S-Plus for the complete data set. 
As stated in Chapter V.A.3, the stepwise regression identified five variables to be 
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SumofSq Means F Value Pr(F) 
Class:PreT est:Rank 1 35.669 35.669 0.30456 0.584032 


Residuals 41 4801./16 117.115 
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Table 19. Analysis of Variance (Complete data set). 


107 


considered for inclusion in the model. This ANOVA was based on those variables and 
considered all interactions. 

Table 20 provides the output for ANOVA of the May 1997 data set. Stepwise 
regression identified six variables in this case. Again, interactions were considered by 


the model. 
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Group 1 7966 7966 180.7974 0.000889 


ExpTime 1 143.066 143.066 3.24/71 0.169348 
SeaTime 1 119.554 119.554 2./134 0.198064 
ESWS 1 235.546 235.546 5.346 0.103828 
PreTest:ExpTime 1 25.097 25.097 0.5696 0.505256 
PreTest:SeaTime 1 2.804 2.804 0.0636 0.817117 


G) 
° 
Cc 
Tb 
g 
= 


mo 
© 
+ 
D |O 
2. 
m 
3 

=_] — 

BAN 


0.012 0.012 0.0003 0.98771 
106.72 106.72 2.4221 0.217492 
xpTime:ESWS 125.069 125.069 2.8386 0.19061 


oO 


SeaTime:ESWS 1 64.645 64.645 1.4672 0.312501 
Group:Education 1 16.942 16.942 0.3845 0.579115 
PreT est:Education 1 28.688 28.688 0.6511 0.478758 


38.076 38.076 


” 
5 
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= 
° 
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BN 
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ESWS: Education 1 9.022 9.022 0.2048 0.681617 
Group:PreTest:ESWS 1 3.96 3.96 0.0899 0.783916 
Residuals 3 132.181 44.06 


Table 20. Analysis of Variance (May 1997 data set). 


Table 21 provides the results of ANOVA for the censored data set. Stepwise 
regression identified six variables, and these were further examined by ANOVA. 
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| Df SumofSq MeanSq F Value  Pr(F) 
Race 1 1848.419 1848.419 2267897 0.0010264 
Group 1 1833.695 1833.695 22.4983 0.0010541 
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ESWS 1 1024.092 1024.092 12.56498 0.0062676 
Race:ESWS 1 131.198 131.198 1.60972 0.2363613 
Race:Class 1 213.854 213.854 2.62385 0.1397207 
IESWS:Class 1 12.227 12.227 0.15002 0.7075132 
Race:Pre.TestScore 1 163.938 163.938 2.01142 0.1898031 
ESWS:Pre.TestScore 1 22.057 22.057 0.27063 0.6154652 
Group:Rank 1 4.797 4.797 0.05885 0.8137546 
Class:Rank 1 10.245 10.245 0.1257 0.7310961 
Race:Group:ESWS 1 452.659 452.659 5.55384 0.0428352 
Race:ESWS:Pre.TestScore 1 35.505 35.505 0.43563 0.5257699 
Race:Class:Pre.TestScore 1 179.472 179.472 2.20201 0.1719847 
ESWS:Class:Pre.TestScore 1 12.792 12.792 0.15695 0.7012067 
Race:Group:Rank 1 19.774 79.774 0.97878 0.34835 
Group:ESWS:Rank 1 455.537 455.537 5.58916 0.0423133 
Race:Class:Rank 1 61.968 61.968 0.76032 0.40589 


Group:Class:Rank 1 201.166 201.166 2.46818 0.1506208 
ESWS:Class:Rank 1 0.262 0.262 0.00322 0.9560002 
Race:Pre.TestScore:Rank 1 19.74 79.74 0.97836 0.3484488 
Group:Pre. TestScore:Rank 1 147.054 147.054 1.80426 0.2120857 
ESWS:Pre.TestScore:Rank 1 208.93 208.93 2.56345 0.143824] 


= 
oe | 


Class:Pre. TestScore:Rank 


9.872 9.872 0.12112 0.735823 
|Group:Class:Pre. T estScore:Rank 8.991 8.991 0.11031 0.7473905) 


1 
[Residuals 9 733.533 81.504 
Table 21. Analysis of Variance (Censored data set). 
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APPENDIX G. POWER FUNCTION GENERATION 


_ This appendix provides the S-Plus function that was used to develop the power 
function curves shown in Chapter V. As shown, this function requires six input values 


and provides a graphical output. 


> FindPower 
function(mean.delta, x, y, data.1, data.2, LvlofSig) 
{ 
# mean.delta is the vector of mean values for which the user wants to calculate the power 
of the test (i.e. mean.delta_seq(-20,20,0.01) 
# x 1S sample variance of treatment group 
# y is sample variance of control group 
# data.1 is treatment group data set dependent variable, a vector 
# data.2 is control group data set dependent variable, a vector 
# LvlofSig is the desired level of statistical significance 

alpha <- 1 - LvlofSig 

for(i in 1:length(mean.delta)) £ 

if{mean.delta[i] > 0) { 
z[i] <- qnorm(alpha) - (mean.delta[i])/(x/length(data.1) + y/ 
length(data.2))*0.5 


j 
else if{mean.delta[i] <0) { 
z[i] <- qnorm(alpha) + (mean. delta[i])/(x/length(data.1) + y/ 
length(data.2))0.5 


3 
else 1f{mean.delta[i] = 0) { 
zZ[i] <- qnorm(alpha) - (mean.delta[1])/(x/length(data.1) + y/ 
length(data.2))0.5 
j 
t <- pnorm(z) 


win. graph() 


plot(mean.delta, 1 - t, type = "1", main = "Power Function for Data", ylab = 
"Power", xlab = "Difference between Data Means") 
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